Compile Error with release 6 (six) in RaspberryPi

  1. last year

    I have a problem in compiling liblfds (version 6) in raspberry pi2 board. This board is using ARMv7. I am running Raspbian as the operating system (linux kernel of this raspbian is 4.1.15). I am compiling on Raspbian with gcc compiler version 4.9. GNU make version is 4.0.
    I get an error in function abstraction_dcas, " asm function has impossible operand".
    could you please advise how to resolve this problem.
    also note please I am allowed to use version 6 only (not 6.0 nor 6.1) as our SW was built on this version.

    Thanks in Advance,

  2. admin

    25 Jan 2016 Administrator
    Edited last year by admin

    Hi, mibrahim.

    Release 6 and 6.*.* support ARM on Linux but was never tested on the Pi (it didn't exist at the time of the release) - so it certainly is possible that there are compile problems on that platform.

    That particular error is to do with the code in the abstraction layer which performs a DWCAS (double-word compare-and-swap). On ARM, that function actually contains assembly code - rather than using the GCC atomic intrinsics - because there is GCC only has instrincs for *single-word* compare-and-swap.

    (In fact, I later realised, if you use an int long long unsigned, which is 64 bit, GCC *will* perform the correct double-word CAS! and that's how 7.0.0 does things, and why ARM64 isn't yet supported - I'll need assembly code for that, and I've not written it yet).

    So it sounds like that assembly code is not working on the Pi, which is unexpected - I would have thought it should be okay.

    Fortunately, I *have* a Pi2 model B, so I should be able now to reproduce the error.

    In general, I profoundly recommend you to move up to release 6.1.1 (although this will still have the same compiler problem - it will not fix the problem in hand). Release 6 *is not right* - it does not correctly use compiler and memory barriers and so in principle is working correctly by chance - even though in practise, the chance of something going wrong is very, very small, and even then requires high data structure load. Nevertheless - there should be *no* chance of error.

    Regarding your continued use of release 6, I may be wrong, but I think there will be two reasons for this; firstly, the work to change the code base over, secondly, testing. With regard to the first, 6.1.1 is API identical to 6. It is only a matter of a global search and replace for variable type names and function names. With regard to the second, you will need to retest, as the code has changed - but the changes *are and only are* the addition of compiler and memory barriers.

    Unless I have blundered and inadvertantly introduced other changes, the addition of compiler and memory barriers *almost certainly cannot break functioning code*. The only way this could occur is if the code was originally working only *because* the compiler or processor were re-ordering when they should not be, and stopping them doing so revealed a fault!

    With regard to the error in hand, I will of course most likely need to make code changes (although there's a small chance it might only be a makefile change) which means you will need a modified version of the code - the fact that the abstraction_dwcas() function has changed would in principle also induce the need for the same testing which would be required by moving to 6.1.1.

    As it is, ideally (and probably anyway, even if you get your own custom version of 6) I would look to release 6.0.2 and 6.1.2, which will contain the fix for Raspberry Pi.

    Either way, what I need to do now is fix the bug. I'll let you know when it's done; it should be today, except I have to figure out a way to get my Pi up and running, and I don't think there's a switch here for the ethernet cable (that's the main problem!)

  3. Hi,
    Thanks alot for your quick and detailed response. I would like to give you also some information that may help while debugging this issue.
    Actually couple of months ago I was using raspbian "wheezy", default compiler for this raspbian-wheezy was gcc-4.3. I tried to compile liblfds but it failed. however we made update for the compiler (from 4.3 to 4.8) then liblfds was compilied successfully.
    This time I am working with raspbian "jessie", I tried then to compile liblfds with updated compiler (4.9) then I got the error that is explained before.
    I tried to compile with version gcc-4.8 as we used in previous raspbian, however this time I got another error message : error: can’t find a register in class ‘CORE_REGS’ while reloading ‘asm’.
    I also tried to compile liblfds 7.0 but I got the same error.
    I hope this info might be helpful.
    Also regarding migrating to work with other version of liblfds, you are right, the code is based over liblfds 6 however it is still big hassle to change lib and functions names in all source files as well as non-arm version of the code is already working on liblfds 6.
    Thanks again for your support,

  4. admin

    25 Jan 2016 Administrator
    Edited last year by admin

    ARM support on GCC has in my experience always been at least a little bit questionable. I remember having a GCC which on ARM *silently emitted no code for the GCC atomic instrinics*.

    The CORE_REGS error occurs when the compiler has run out of registers - the DWCAS assembly code must use a lot of registers, and it's an error I remember seeing on x86 during the development - but the compiler I think should in theory simply at that point move stuff out of the registers so that the inline assembly *will* work; unless the code really is trying to use a register which is not available (frame pointer, say) then it ought always to work.

    I am alarmed 7.0.0 however shows the same error - this is NOT expected and should NOT occur, as it is using the GCC inline intrinsics! this could conceivably mean there is a real compiler bug here. I think I was using 4.9.2 on my Pi2, but I can't be sure I remember correctly (I'll check once I've got it going). However, I wonder - it might be GCC is doing something which genuinely consumes additional register(s) - like frame pointers, and I'm vaguely remembering now shared objects might need an extra register too? - and maybe this is enough to cause the CORE_REGS problem.

    On the other hand, I built every variant on my Pi2, and the matching test programme, and ran them all, so the makefile shipped with the library should it and of itself work, at least with the compiler version I had.

    Regarding code change over - I think it will be a matter of only a small script to make the change, which wil issue a couple of recursive sed commands. The problem is knowing what to replace with what! In fact, I should have written that script; I am remiss not to have done so. I will write it, after I've sorted out the current problem, and make it generally available on the site.

  5. I tried again with liblfds7.0.0. currently it compiles successfully. well, I don't know what makes it works, the only thing that I changed in my system is installing protobuf library which is installed along with (autoconf-automake-libtool) libraries so I have no idea actually if this has effect or not.

  6. admin

    25 Jan 2016 Administrator
    Edited last year by admin

    Still trying to SSH to the Pi, using a cable directly from my laptop. Usual bloody Linux catastrophe. Try and do anything and it's a recursive descent into problem after problem after problem, each of which stops you from doing what you were trying to do before.

  7. admin

    26 Jan 2016 Administrator

    Finally. Started yesterday at 17:00, it's now 13:30. I have SSHed to the Pi, over a direct ethernet connection. Recursive descent into problems, none of which were solved - DHCP relay doesn't work, DHCP server doesn't work, I think also the SD card image was damaged more than once, and I'm not certain why (and the laptop error messages make it look like a hardware problem) - I had to rewrite it twice. In the end I gave up with everything and just went back to a static IP - and that took about two hours to get working as well!

  8. admin

    26 Jan 2016 Administrator

    BTW do you mind if I modify the thread title to mention that the compile error is in liblfds 6? otherwise it gives the wrong impression to people coming to get the current version.

  9. yeah sure, feel free to change it.

  10. admin

    26 Jan 2016 Administrator
    Edited last year by admin

    So, I took the release 6 code and tried to compile it on my Pi2.

    GCC version 4.9.2, and this is the uname -a;

    Linux raspberrypi 4.1.7-v7+ #817 SMP PREEMPT Sat Sep 19 15:32:00 BST 2015 armv7l GNU/Linux

    I compiled the library and got the "impossible operand" error.

    I then took the 7.0.0 macro for DWCAS, which uses the GCC intrinsic (the new style instrincs, so you need GCC 4.7.3 or greater), added it to the DWCAS file, removed the body of the 6 DWCAS function and replaced it with a call to that macro - and that compiled, and the test programmes passes on debug and release.

    The code is below - replace the matching code in "abstraction_dcas.c" (line 259 to the end of the file). The formatting looks odd below because the macro has long lines. It'll look fine if you copy and paste into a text editor.

    #if (defined __arm__ && __GNUC__)
      /* TRD : any OS on any ARM with GCC
               Remember however we need to set into compare the original value of destination.
               __arm__   indicates ARM
               __GNUC__  indicates GCC
      #define LFDS710_PAL_ATOMIC_DWCAS( pointer_to_destination, pointer_to_compare, pointer_to_new_destination, cas_strength, result )                                                                                                                                                          \
      {                                                                                                                                                                                                                                                                                         \
        (result) = (char unsigned) __atomic_compare_exchange_n( (int long long unsigned volatile *) (pointer_to_destination), (int long long unsigned *) (pointer_to_compare), *(int long long unsigned *) (pointer_to_new_destination), (cas_strength), __ATOMIC_RELAXED, __ATOMIC_RELAXED );  \
      INLINE unsigned char abstraction_dcas( volatile atom_t *destination, atom_t *exchange, atom_t *compare )
        char unsigned
        assert( destination != NULL );
        assert( exchange != NULL );
        assert( compare != NULL );
        LFDS710_PAL_ATOMIC_DWCAS( destination, compare, exchange, 0, result );
        return( result );
  11. admin

    26 Jan 2016 Administrator

    I can't find how to change the thread title any more =-) I'm sure I saw it yesterday!

  12. admin

    29 Jan 2016 Administrator

    Hej, did it work? is it okay? is the requirement for 4.7.3 or better okay? the other macro, for 4.1.2 or greater, can be used instead.

  13. Deleted last year by mibrahim
  14. Edited last year by mibrahim

    Sorry for my late response , I was involved actually in another project last week. well I tried this modification and THANKS alot, it has been compiled. I have the following observations:

    • I replaced arm section of abstraction_dcas.c file with the one that you provided.
    • I ran make -f makefile.linux -sorel PREFIX=/path/to/target/directory in order to get header in "include" and .a output in "lib" directory

    Now what happened is:

    • no response was observed in terminal interface (nothing indicates if successfully finished or failed).
    • I can see object abstraction_dcas.o now is created in obj directory.
    • I found liblfds.a in 'bin' directory (not the directory specified in PREFIX) (so I will copy from bin to lib directory no problem in that)
    • I tried to run make clean but this is not acceptable, it tells me "no rule for make clean ..".

    So we can say that it works for me now, I just wanted to add my observations , I don't know if you have the same or not.
    I really appreciate your support and help for making this solved.


  15. admin

    1 Feb 2016 Administrator
    Edited last year by admin

    Well, I think I'm lucky you asked for help - it's not often that there is a chance to demonstrate support, and people in general are encouraged in their use of the library by seeing that support is given.

    So, the makefile - it doesn't know about PREFIX. I've become aware, over time, that there are a number of conventions for makefiles, such as CFLAGS, and now you mention it, PREFIX. However, I still don't know really what they are, and I didn't know at all when release 6 was made. So the makefile simply isn't behaving in the way you're expecting, in response to the environment variables you're setting - it isn't looking at PREFIX at all, it's just doing its thing.

    There is a clean target though, and that should certainly work! and Im not sure what you mean by no response in the terminal interface? you mean when you compile?

  16. Deleted last year by mibrahim
  17. Deleted last year by mibrahim
  18. Edited last year by mibrahim

    Well, I made some trials this morning. I tried to compile without any modifications in abstract_dcas file, so let me add some updates.
    1- this 'silence' was because of I am running the following
    make -f makefile.linux -sorel
    but I tried running this: make -f makefile.linux sorel it started to compile and displaying what is happening during compile process.
    2- the surprising thing for me today is gcc-4.9 is able to compile " make -f makefile.linux sorel" but not able to compile "make -f makefile.linux -sorel" (that is because -sorel is not recognized by the make file so it is treated as default option "dbg").
    3- then I ran "make -f makefile.linux clean" then clean is now working (previously I was running "make -f makefile.linux -clean")
    4- then I ran "make -f makefile.linux sodbg". I found that gcc-4.9 is not able to compile (gives impossible operand error) but only gcc-4.4 is able to compile it.

    So I concluded that gcc-4.9 is able to compile for "rel" but not for "dbg". If I want to run for "dbg" using gcc-4.9, I should modify abstract_dcas file as you suggested.

    I have no problems so far, I understand now how to work around this compile process. just wanted to add this update

  19. admin

    2 Feb 2016 Administrator
    Edited last year by admin

    You found the issue - the makefile expect the targets just to be the name of the target, "arrel" or "sodbg", etc; without a leading minus sign. It's the same for "clean".

    With the original code, without the fix from earlier in this thread, it is understandable that the "sodbg" build fails while the others work. I think that build uses two more registers, just to work normally, than the (for example) "arrel" build. Where so many registers are used for the DWCAS code on ARM, those two extra registers are just too much - at least, with GCC 4.9; 4.4 works. Or maybe it doesn't work, but it hasn't noticed! scary thought...

    When the release was made, every build variant was built, and the test and benchmark run - so with the GCC I had at the time, it passed.

    With the fix from earlier on in this thread, I think you're okay now, with "sodbg", and with GCC 4.9, is that right?

  20. Edited last year by mibrahim

    yes, I am using this fix in order to compile sodbg with gcc-4.9


or Sign Up to reply!