gh-Cyan4973/xxHash

Watch

Fork 0

Test fails on Power with GCC 10.3.0 #596

New issue

Closed

opened 2021-11-15 00:51:41 -06:00 by mpe · 1 comment

mpe commented

2021-11-15 00:51:41 -06:00

(Migrated from github.com)

On Power8/Power9 (ppc64le) I see the tests fail when building with GCC 10.3.0:

$ make test
...
./xxhsum -bi0
xxhsum 0.8.1 by Yann Collet
compiled as 64-bit ppc64 + POWER8 vector little endian with GCC 10.3.0
Error: 64-bit hash test 100: Internal sanity check failed!
Got 0xBB9D481D33CB87AAULL, expected 0xCDEB804D65C6DEA4ULL.
Note: If you modified the hash functions, make sure to either update the values
or temporarily recompile with XSUM_NO_TESTS=1.
make: *** [Makefile:196: check] Error 1

The failure is always the same.

If I disable strict aliasing with -fno-strict-aliasing all tests pass.

I see there was a pull request a while back from @easyaspi314 to fix some VSX issues: https://github.com/Cyan4973/xxHash/pull/433

But it was never merged in any form as far as I can see.

If I take the latter two commits from that pull request:

https://github.com/Cyan4973/xxHash/pull/433/commits/2e9033d06d98fb321f42f633189a4a53f22318f5
https://github.com/Cyan4973/xxHash/pull/433/commits/cf1988f2faa82bb99a44a998e67b0d74b93b0d24

And apply them on HEAD (227fd61077), then the tests pass again.

I have also observed rsync failing to transfer files to Power machines when using xxh128, if I rebuild libxxhash.so using the above patches and LD_PRELOAD it then rsync works again.

Should I submit a pull request with those VSX fixes rebased onto HEAD? I could test it on Power8/Power9 big and little endian, with various compilers.

On Power8/Power9 (ppc64le) I see the tests fail when building with GCC 10.3.0: ``` $ make test ... ./xxhsum -bi0 xxhsum 0.8.1 by Yann Collet compiled as 64-bit ppc64 + POWER8 vector little endian with GCC 10.3.0 Error: 64-bit hash test 100: Internal sanity check failed! Got 0xBB9D481D33CB87AAULL, expected 0xCDEB804D65C6DEA4ULL. Note: If you modified the hash functions, make sure to either update the values or temporarily recompile with XSUM_NO_TESTS=1. make: *** [Makefile:196: check] Error 1 ``` The failure is always the same. If I disable strict aliasing with `-fno-strict-aliasing` all tests pass. I see there was a pull request a while back from @easyaspi314 to fix some VSX issues: https://github.com/Cyan4973/xxHash/pull/433 But it was never merged in any form as far as I can see. If I take the latter two commits from that pull request: https://github.com/Cyan4973/xxHash/pull/433/commits/2e9033d06d98fb321f42f633189a4a53f22318f5 https://github.com/Cyan4973/xxHash/pull/433/commits/cf1988f2faa82bb99a44a998e67b0d74b93b0d24 And apply them on HEAD (227fd610778a82c9eab9674420d1704b4574b14a), then the tests pass again. I have also observed rsync failing to transfer files to Power machines when using xxh128, if I rebuild libxxhash.so using the above patches and LD_PRELOAD it then rsync works again. Should I submit a pull request with those VSX fixes rebased onto HEAD? I could test it on Power8/Power9 big and little endian, with various compilers.

👍 2

t-mat commented

2021-11-15 02:37:09 -06:00

(Migrated from github.com)

Hi @mpe, thanks for the report.

I've confirmed your issue with QEMU (qemu-ppc64le) and gcc-10 cross compiler. Here're some notes about this issue:

This issue happens when ppc64 user uses gcc-10 or greater.
gcc-9 or earlier version don't cause this issue.
It also means our current test works properly, but need to include newer cross compiler to catch this issue.

Should I submit a pull request with those VSX fixes rebased onto HEAD?

I think it'd be great if you can rebase and send pull request with these commits.

After that, I'd like to send another PR for test script which adds QEMU test with gcc-10 for ppc64.

Setup QEMU and GCC cross compiler

cd
git clone https://github.com/Cyan4973/xxHash.git xxHash-issue-596
cd xxHash-issue-596
sudo apt-get update
sudo apt-get install gcc-multilib g++-multilib qemu-utils qemu-user-static
sudo apt-get install gcc-9-powerpc64le-linux-gnu qemu-system-ppc
sudo apt-get install gcc-10-powerpc64le-linux-gnu

Test with gcc-10 (fail)

XCC=powerpc64le-linux-gnu-gcc-10
XEMU=qemu-ppc64le-static
CPPFLAGS="-DXXH_VECTOR=XXH_VSX" CFLAGS="-O3 -maltivec -mvsx -mpower8-vector -mcpu=power8" LDFLAGS="-static" CC=$XCC RUN_ENV=$XEMU make clean check

...
qemu-ppc64le-static ./xxhsum -bi0
xxhsum 0.8.1 by Yann Collet
compiled as 64-bit ppc64 + POWER8 vector little endian with GCC 10.3.0
Error: 64-bit hash test 100: Internal sanity check failed!
Got 0xBB9D481D33CB87AAULL, expected 0xCDEB804D65C6DEA4ULL.
Note: If you modified the hash functions, make sure to either update the values
or temporarily recompile with XSUM_NO_TESTS=1.
make: *** [Makefile:196: check] Error 1

$ powerpc64le-linux-gnu-gcc-10 --version
powerpc64le-linux-gnu-gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0

Test with gcc-9 (pass)

XCC=powerpc64le-linux-gnu-gcc-9
XEMU=qemu-ppc64le-static
CPPFLAGS="-DXXH_VECTOR=XXH_VSX" CFLAGS="-O3 -maltivec -mvsx -mpower8-vector -mcpu=power8" LDFLAGS="-static" CC=$XCC RUN_ENV=$XEMU make clean check

...
 .......   checks completed successfully   .......

$ powerpc64le-linux-gnu-gcc-9 --version
powerpc64le-linux-gnu-gcc-9 (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0

Hi @mpe, thanks for the report. I've confirmed your issue with QEMU (`qemu-ppc64le`) and `gcc-10` cross compiler. Here're some notes about this issue: - This issue happens when ppc64 user uses `gcc-10` or greater. - `gcc-9` or earlier version don't cause this issue. - It also means our current test works properly, but need to include newer cross compiler to catch this issue. > Should I submit a pull request with those VSX fixes rebased onto HEAD? I think it'd be great if you can rebase and send pull request with these commits. After that, I'd like to send another PR for test script which adds QEMU test with `gcc-10` for ppc64. ## Setup QEMU and GCC cross compiler ``` cd git clone https://github.com/Cyan4973/xxHash.git xxHash-issue-596 cd xxHash-issue-596 sudo apt-get update sudo apt-get install gcc-multilib g++-multilib qemu-utils qemu-user-static sudo apt-get install gcc-9-powerpc64le-linux-gnu qemu-system-ppc sudo apt-get install gcc-10-powerpc64le-linux-gnu ``` ## Test with gcc-10 (fail) ``` XCC=powerpc64le-linux-gnu-gcc-10 XEMU=qemu-ppc64le-static CPPFLAGS="-DXXH_VECTOR=XXH_VSX" CFLAGS="-O3 -maltivec -mvsx -mpower8-vector -mcpu=power8" LDFLAGS="-static" CC=$XCC RUN_ENV=$XEMU make clean check ``` ``` ... qemu-ppc64le-static ./xxhsum -bi0 xxhsum 0.8.1 by Yann Collet compiled as 64-bit ppc64 + POWER8 vector little endian with GCC 10.3.0 Error: 64-bit hash test 100: Internal sanity check failed! Got 0xBB9D481D33CB87AAULL, expected 0xCDEB804D65C6DEA4ULL. Note: If you modified the hash functions, make sure to either update the values or temporarily recompile with XSUM_NO_TESTS=1. make: *** [Makefile:196: check] Error 1 ``` ``` $ powerpc64le-linux-gnu-gcc-10 --version powerpc64le-linux-gnu-gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0 ``` ## Test with gcc-9 (pass) ``` XCC=powerpc64le-linux-gnu-gcc-9 XEMU=qemu-ppc64le-static CPPFLAGS="-DXXH_VECTOR=XXH_VSX" CFLAGS="-O3 -maltivec -mvsx -mpower8-vector -mcpu=power8" LDFLAGS="-static" CC=$XCC RUN_ENV=$XEMU make clean check ``` ``` ... ....... checks completed successfully ....... ``` ``` $ powerpc64le-linux-gnu-gcc-9 --version powerpc64le-linux-gnu-gcc-9 (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 ```

👍 2

No milestone

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

gh-Cyan4973/xxHash#596

No description provided.

Rows
Columns