Xcode4, libtomcrypt, LLVM, asm (unknown version of libtomcrypt)

lonegraywolf2000 commented

2011-06-06 16:38:03 -05:00

(Migrated from github.com)

Good insert time zone here everyone. I represent the StepMania 5/StepMania sm-ssc group from Google Code, and I am trying to work on bringing our project up to date for Xcode 4 on the Mac OS X side. Our project uses libtommath and libtomcrypt. Unfortunately, I do not know what version it is, which makes this issue a little hard to figure out.

The old environment we used, Xcode 3, used GCC 4.0 for historical reasons. With the move to Xcode 4, we wish to take advantage of a faster, better compiler: specifically, LLVM 2.0 (also known as Clang). Unfortunately, that better, faster compiler is not working due to this one small issue.

In tomcrypt_macros.h, there are two sets of ROLc and RORc static inline functions that both contain asm code. The core part of one of them is as follows:

static inline __attribute__((always_inline)) unsigned ROLc(unsigned word, const int i)
{
   asm ("roll %2,%0"
      :"=r" (word)
      :"0" (word),"I" (i));
   return word;
}

The compiler complains on aes.c, md5.c, and sha1.c of this message:

fatal error: error in backend: Invalid operand for inline asm constraint 'I'!

That function, and three others just like it, are the only occasions where there is the 'I' constraint.

If we try to use GCC 4.2 for libtomcrypt and Clang for the rest of our libraries (including libtommath), this eventually comes up on linking.

Ld ../sm-ssc.app/Contents/MacOS/sm-ssc normal i386
    cd /Users/wolfman2000/Repositories/sm-ssc/Xcode
    setenv MACOSX_DEPLOYMENT_TARGET 10.5
    /Developer/usr/bin/clang++ -arch i386 -isysroot /Developer/SDKs/MacOSX10.5.sdk
 -L/Users/wolfman2000/Repositories/sm-ssc/Xcode/..
 -L/Users/wolfman2000/Repositories/sm-ssc/Xcode/build/Debug
 -L/Users/wolfman2000/Repositories/sm-ssc/Xcode/Libraries
 -L/Users/wolfman2000/Repositories/sm-ssc/Xcode/../extern/mad-0.15.1b/msvc++/Release
 -F/Users/wolfman2000/Repositories/sm-ssc/Xcode/..
 -filelist /Users/wolfman2000/Repositories/sm-ssc/Xcode/build/stepmania_xcode4.build/Debug/stepmania.build/Objects-normal/i386/sm-ssc.LinkFileList
 -mmacosx-version-min=10.5 -dead_strip -no_dead_strip_inits_and_terms
 -framework Cocoa -framework Carbon -framework IOKit
 -framework QuickTime -framework OpenGL -framework CoreFoundation
 -framework AudioToolbox -framework AudioUnit -framework CoreAudio
 -framework vecLib -ljpeg_link -logg_link -ltheora_link -lvorbis_link
 -llua-5.1 -ltomcrypt -ltommath -lz_link -lffmpeg_link -lmad-sm5
 -lpng-1.5.1sm0.1 -lglew -lpcre -o /Users/wolfman2000/Repositories/sm-ssc/Xcode/../sm-ssc.app/Contents/MacOS/sm-ssc

ld: illegal text reloc to 
__ZSt7find_ifIN9__gnu_cxx17__normal_iteratorIPN9StdString7CStdStrIcEESt6vectorIS4_SaIS4_EEEEPFbRKS4_EET_SE_SE_T0_St26random_access_iterator_tag from /Users/wolfman2000/Repositories/sm-ssc/Xcode/build/stepmania_xcode4.build
/Debug/stepmania.build/Objects-normal/i386/RageUtil.o in 
__ZSt7find_ifIN9__gnu_cxx17__normal_iteratorIPN9StdString7CStdStrIcEESt6vectorIS4_SaIS4_EEEEPFbRKS4_EET_SE_SE_T0_St26random_access_iterator_tag for architecture i386
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Command /Developer/usr/bin/clang++ failed with exit code 1

I admit I am at a loss as to how to properly solve this.

The code we use for libtomcrypt is included in our repository (Mercurial required). We haven't moved it to our externs yet, so it's located in src/libtomcrypt/src. The spcific branch that I am trying to make work is the XCode4 branch, but I believe any attempts at compiling libtomcrypt via Clang will fail, not just my end. If requested, I will upload just the libtomcrypt part at the location of your choice.

Jason "Wolfman2000" Felds

Good _insert time zone here_ everyone. I represent the StepMania 5/StepMania sm-ssc group from [Google Code](http://code.google.com/p/sm-ssc/), and I am trying to work on bringing our project up to date for Xcode 4 on the Mac OS X side. Our project uses libtommath and libtomcrypt. Unfortunately, I do not know what version it is, which makes this issue a little hard to figure out. The old environment we used, Xcode 3, used GCC 4.0 for historical reasons. With the move to Xcode 4, we wish to take advantage of a faster, better compiler: specifically, LLVM 2.0 (also known as Clang). Unfortunately, that better, faster compiler is not working due to this one small issue. In `tomcrypt_macros.h`, there are two sets of ROLc and RORc static inline functions that both contain asm code. The core part of one of them is as follows: ``` c static inline __attribute__((always_inline)) unsigned ROLc(unsigned word, const int i) { asm ("roll %2,%0" :"=r" (word) :"0" (word),"I" (i)); return word; } ``` The compiler complains on aes.c, md5.c, and sha1.c of this message: `fatal error: error in backend: Invalid operand for inline asm constraint 'I'!` That function, and three others just like it, are the only occasions where there is the 'I' constraint. If we try to use GCC 4.2 for libtomcrypt and Clang for the rest of our libraries (including libtommath), this eventually comes up on linking. ``` Ld ../sm-ssc.app/Contents/MacOS/sm-ssc normal i386 cd /Users/wolfman2000/Repositories/sm-ssc/Xcode setenv MACOSX_DEPLOYMENT_TARGET 10.5 /Developer/usr/bin/clang++ -arch i386 -isysroot /Developer/SDKs/MacOSX10.5.sdk -L/Users/wolfman2000/Repositories/sm-ssc/Xcode/.. -L/Users/wolfman2000/Repositories/sm-ssc/Xcode/build/Debug -L/Users/wolfman2000/Repositories/sm-ssc/Xcode/Libraries -L/Users/wolfman2000/Repositories/sm-ssc/Xcode/../extern/mad-0.15.1b/msvc++/Release -F/Users/wolfman2000/Repositories/sm-ssc/Xcode/.. -filelist /Users/wolfman2000/Repositories/sm-ssc/Xcode/build/stepmania_xcode4.build/Debug/stepmania.build/Objects-normal/i386/sm-ssc.LinkFileList -mmacosx-version-min=10.5 -dead_strip -no_dead_strip_inits_and_terms -framework Cocoa -framework Carbon -framework IOKit -framework QuickTime -framework OpenGL -framework CoreFoundation -framework AudioToolbox -framework AudioUnit -framework CoreAudio -framework vecLib -ljpeg_link -logg_link -ltheora_link -lvorbis_link -llua-5.1 -ltomcrypt -ltommath -lz_link -lffmpeg_link -lmad-sm5 -lpng-1.5.1sm0.1 -lglew -lpcre -o /Users/wolfman2000/Repositories/sm-ssc/Xcode/../sm-ssc.app/Contents/MacOS/sm-ssc ld: illegal text reloc to __ZSt7find_ifIN9__gnu_cxx17__normal_iteratorIPN9StdString7CStdStrIcEESt6vectorIS4_SaIS4_EEEEPFbRKS4_EET_SE_SE_T0_St26random_access_iterator_tag from /Users/wolfman2000/Repositories/sm-ssc/Xcode/build/stepmania_xcode4.build /Debug/stepmania.build/Objects-normal/i386/RageUtil.o in __ZSt7find_ifIN9__gnu_cxx17__normal_iteratorIPN9StdString7CStdStrIcEESt6vectorIS4_SaIS4_EEEEPFbRKS4_EET_SE_SE_T0_St26random_access_iterator_tag for architecture i386 clang: error: linker command failed with exit code 1 (use -v to see invocation) Command /Developer/usr/bin/clang++ failed with exit code 1 ``` I admit I am at a loss as to how to properly solve this. The code we use for libtomcrypt is included in [our repository](http://code.google.com/p/sm-ssc/source/checkout) (Mercurial required). We haven't moved it to our externs yet, so it's located in `src/libtomcrypt/src`. The spcific branch that I am trying to make work is the `XCode4` branch, but I believe any attempts at compiling libtomcrypt via Clang will fail, not just my end. If requested, I will upload just the libtomcrypt part at the location of your choice. Jason "Wolfman2000" Felds

sjaeckel commented

2011-06-14 04:58:08 -05:00

(Migrated from github.com)

as far as I can see you're using 1.16
[edit]but you seem to have modified some files. e.g. the "attribute((always_inline))" isn't in the sources of the standard 1.16 tomcrypt_macros.c[/edit]

as far as I can see you're using 1.16 [edit]but you seem to have modified some files. e.g. the "**attribute**((always_inline))" isn't in the sources of the standard 1.16 tomcrypt_macros.c[/edit]

lonegraywolf2000 commented

2011-06-14 10:43:15 -05:00

(Migrated from github.com)

I wasn't the one that made those attribute changes. Still...good to see that there's something.

Does the latest version use those attributes as well?

I wasn't the one that made those attribute changes. Still...good to see that there's something. Does the latest version use those attributes as well?

donaldguy commented

2011-10-21 06:27:04 -05:00

(Migrated from github.com)

I am presently attempting to get Stempania 3.9 to build on Lion (which is more of a challenge, I suspect) and am seeing this error. Though extra bizarrely I seem to only see it when I execute the build from the XCode GUI. If I use xcodebuild from the command line it seems to build cleanly (though I haven't gotten the rest to build enough to have any hope of testing linking yet).

donaldguy commented

2011-10-21 06:48:46 -05:00

(Migrated from github.com)

It seems like the difference, at least for 3.9 may be that the GUI seems to be hitting the "Debug" configuration, whereas xcodebuild is using the "Native" configuration. I have't looked into it deeper than that.

sporksmith commented

2012-06-06 20:51:05 -05:00

(Migrated from github.com)

I ran into this issue as well. The workaround I ended up doing was to add #define LTC_NO_ROLC to tomcrypt_custom.h.

I think the issue is that the ROLc and RORc functions attempt to emit rotate instructions with a hard-coded literal, from the parameter i. This is what the asm constraint "I" means. Since the functions are specified as inline, the compiler could end up replacing the (i) parameter in the asm with a literal, in which case it works fine.

ROLc is defined as:

static inline unsigned ROLc(unsigned word, const int i)
{
   asm ("roll %2,%0"
      :"=r" (word)
      :"0" (word),"I" (i));
   return word;
}

I believe the intent is for something like y = ROLc(x, 3) to get rewritten as follows before getting handed off to the assembler:

asm ("roll %2,%0"
  :"=r" (x)
  :"0" (x),"I" (3));
y = x;

However, if the function doesn't get inlined, or the compiler doesn't optimize (i) to a literal constant as a result of inlining, then the constraint "I" will indeed be invalid.

It sounds like someone tried to address this issue in your codebase by forcing the compiler to always inline. Unfortunately this won't work if the attribute gets ignored, or if the compiler doesn't do the further optimization of replacing (i) with the literal parameter.

I think that the most portable fix would be to make ROLc and RORc macros instead of functions.

I ran into this issue as well. The workaround I ended up doing was to add `#define LTC_NO_ROLC` to `tomcrypt_custom.h`. I think the issue is that the `ROLc` and `RORc` functions attempt to emit rotate instructions with a hard-coded literal, from the parameter `i`. This is what the asm constraint `"I"` means. Since the functions are specified as `inline`, the compiler could end up replacing the `(i)` parameter in the asm with a literal, in which case it works fine. `ROLc` is defined as: ``` static inline unsigned ROLc(unsigned word, const int i) { asm ("roll %2,%0" :"=r" (word) :"0" (word),"I" (i)); return word; } ``` I believe the intent is for something like `y = ROLc(x, 3)` to get rewritten as follows before getting handed off to the assembler: ``` asm ("roll %2,%0" :"=r" (x) :"0" (x),"I" (3)); y = x; ``` However, if the function doesn't get inlined, or the compiler doesn't optimize (i) to a literal constant as a result of inlining, then the constraint `"I"` will indeed be invalid. It sounds like someone tried to address this issue in your codebase by forcing the compiler to always inline. Unfortunately this won't work if the attribute gets ignored, or if the compiler doesn't do the further optimization of replacing `(i)` with the literal parameter. I think that the most portable fix would be to make `ROLc` and `RORc` macros instead of functions.

sporksmith commented

2012-06-06 21:25:49 -05:00

(Migrated from github.com)

I thought I'd go ahead and give a quick try for redefining ROLc and RORc as macros. The tricky part is that an asm block is a statement, not an expression. The only way I can think to do it is using statement expressions, which are supported by both gcc and clang, but are not ansi c.

We also need to copy the parameter to avoid changing its value and to handle the case where the parameter isn't an lvalue.

Anyways, the following compiles, but I haven't tested for correctness:

#define ROLc(word, i)                           \                                                            
  ({ unsigned __rotd;                           \                                                            
    __rotd = (word);                            \                                                            
    asm ("roll %2,%0"                           \                                                                     :"=r" (__rotd)                         \                                                            
         :"0" (__rotd),"I" (i));                \                                                            
    __rotd; })                                                                                               

#define RORc(word, i)                           \                                                            
  ({ unsigned __rotd;                           \                                                            
    __rotd = (word);                            \                                                            
    asm ("rorl %2,%0"                           \                                                            
         :"=r" (__rotd)                         \
         :"0" (__rotd),"I" (i));                \
    __rotd; })

I thought I'd go ahead and give a quick try for redefining `ROLc` and `RORc` as macros. The tricky part is that an asm block is a statement, not an expression. The only way I can think to do it is using [statement expressions](http://gcc.gnu.org/onlinedocs/gcc/Statement-Exprs.html), which are supported by both gcc and clang, but are not ansi c. We also need to copy the parameter to avoid changing its value and to handle the case where the parameter isn't an lvalue. Anyways, the following compiles, but I haven't tested for correctness: ``` #define ROLc(word, i) \ ({ unsigned __rotd; \ __rotd = (word); \ asm ("roll %2,%0" \ :"=r" (__rotd) \ :"0" (__rotd),"I" (i)); \ __rotd; }) #define RORc(word, i) \ ({ unsigned __rotd; \ __rotd = (word); \ asm ("rorl %2,%0" \ :"=r" (__rotd) \ :"0" (__rotd),"I" (i)); \ __rotd; }) ```

rofl0r commented

2012-06-07 03:46:15 -05:00

(Migrated from github.com)

how about using the old trick: do { code_here(); } while(0)

sporksmith commented

2012-06-07 05:34:00 -05:00

(Migrated from github.com)

how about using the old trick: do { code_here(); } while(0)

That trick only works to replace a function that returns void. This one needs to return the rotated value. Here's an example call site from aes.c:

rk[12] = rk[ 4] ^ setup_mix(RORc(temp, 8));

(edited for clarification)

> how about using the old trick: do { code_here(); } while(0) That trick only works to replace a function that returns void. This one needs to return the rotated value. Here's an example call site from aes.c: ``` rk[12] = rk[ 4] ^ setup_mix(RORc(temp, 8)); ``` (edited for clarification)

sporksmith commented

2012-06-07 05:42:30 -05:00

(Migrated from github.com)

Incidentally, I think the reason I'm hitting this bug is because I'm compiling with -O0, causing gcc to skip the inlining. I'm including tomcrypt as part of a larger project build, from which the tomcrypt build inherits some CFLAGS. i.e., the bug can be replicated with gcc on the stock code base by:

make CFLAGS="-O0 -I./src/headers -DLTC_SOURCE"

Incidentally, I think the reason I'm hitting this bug is because I'm compiling with -O0, causing gcc to skip the inlining. I'm including tomcrypt as part of a larger project build, from which the tomcrypt build inherits some CFLAGS. i.e., the bug can be replicated with gcc on the stock code base by: ``` make CFLAGS="-O0 -I./src/headers -DLTC_SOURCE" ```

sjaeckel commented

2013-07-04 08:23:15 -05:00

(Migrated from github.com)

@sporksmith did you try to define the function with __attribute__((always_inline))?

@sporksmith did you try to define the function with `__attribute__((always_inline))`?

sporksmith commented

2013-07-04 09:51:59 -05:00

(Migrated from github.com)

I don't remember whether we tried __attribute__((always_inline)) or not. I imagine it would work, if gcc honors that attribute even at -O0.

We ended up solving it by just not having the tomcrypt build inherit the CFLAGS from the project build. I.e., let tomcrypt compile at -O2, in which case the functions gets inlined and this bug doesn't surface.

I don't remember whether we tried `__attribute__((always_inline))` or not. I imagine it would work, if gcc honors that attribute even at `-O0`. We ended up solving it by just not having the tomcrypt build inherit the `CFLAGS` from the project build. I.e., let tomcrypt compile at `-O2`, in which case the functions gets inlined and this bug doesn't surface.

MMI commented

2016-07-28 12:12:43 -05:00

(Migrated from github.com)

Proposed solution in this pull request

Proposed solution in this [pull request](https://github.com/libtom/libtomcrypt/pull/125)

karel-m commented

2017-03-01 15:24:04 -06:00

(Migrated from github.com)

pull/125 was closed, can we also close this issue?

Rows
Columns

Xcode4, libtomcrypt, LLVM, asm (unknown version of libtomcrypt) #2