Streaming much slower than XXH3_64b for AVX 2 on MSVC

pstadermann commented

2021-03-17 08:18:01 -05:00

(Migrated from github.com)

When running "xxhsum -b" I get the following results for SSE2:

xxh-msvc 0.8.0 by Yann Collet
compiled as 64-bit x86_64 + SSE2 little endian with MSVC 19.28.29336.00
Sample of 100 KB...
5#XXH3_64b : 102400 -> 265217 it/s (25900.1 MB/s)
17#XXH3_stream : 102400 -> 224415 it/s (21915.5 MB/s)

With AVX2 the performance for non-streaming greatly increases, but streaming does not:

xxh-msvc 0.8.0 by Yann Collet
compiled as 64-bit x86_64 + AVX2 little endian with MSVC 19.28.29336.00
Sample of 100 KB...
5#XXH3_64b : 102400 -> 613544 it/s (59916.4 MB/s)
17#XXH3_stream : 102400 -> 219870 it/s (21471.7 MB/s)

Is this expected or am I doing something wrong (compiler switches, ...)?

Tested for master and v0.8.0
CPU: Ryzen 3900x
Compiler switches:
/permissive- /ifcOutput "x64\Release" /GS /GL /W3 /Gy /Zc:wchar_t /Zi /Gm- /O2 /sdl /Fd"x64\Release\vc142.pdb" /Zc:inline /fp:precise /D "NDEBUG" /D "_CONSOLE" /D "_UNICODE" /D "UNICODE" /errorReport:prompt /WX- /Zc:forScope /arch:AVX2 /Gd /Oi /MD /FC /Fa"x64\Release" /EHsc /nologo /Fo"x64\Release" /Fp"x64\Release\xxh-msvc.pch" /diagnostics:column

When running "xxhsum -b" I get the following results for SSE2: xxh-msvc 0.8.0 by Yann Collet compiled as 64-bit x86_64 + SSE2 little endian with MSVC 19.28.29336.00 Sample of 100 KB... 5#XXH3_64b : 102400 -> 265217 it/s (25900.1 MB/s) 17#XXH3_stream : 102400 -> 224415 it/s (21915.5 MB/s) With AVX2 the performance for non-streaming greatly increases, but streaming does not: xxh-msvc 0.8.0 by Yann Collet compiled as 64-bit x86_64 + AVX2 little endian with MSVC 19.28.29336.00 Sample of 100 KB... 5#XXH3_64b : 102400 -> 613544 it/s (59916.4 MB/s) 17#XXH3_stream : 102400 -> 219870 it/s (21471.7 MB/s) Is this expected or am I doing something wrong (compiler switches, ...)? Tested for master and v0.8.0 CPU: Ryzen 3900x Compiler switches: /permissive- /ifcOutput "x64\Release\" /GS /GL /W3 /Gy /Zc:wchar_t /Zi /Gm- /O2 /sdl /Fd"x64\Release\vc142.pdb" /Zc:inline /fp:precise /D "NDEBUG" /D "_CONSOLE" /D "_UNICODE" /D "UNICODE" /errorReport:prompt /WX- /Zc:forScope /arch:AVX2 /Gd /Oi /MD /FC /Fa"x64\Release\" /EHsc /nologo /Fo"x64\Release\" /Fp"x64\Release\xxh-msvc.pch" /diagnostics:column

👍 1

Cyan4973 commented

2021-03-17 12:05:01 -05:00

(Migrated from github.com)

I do not observe this impact on my laptop:

compiled as 64-bit x86_64 + AVX2 little endian with Apple Clang 11.0.3 (clang-1103.0.32.62)
 5#XXH3_64b                      :     102400 ->   294153 it/s (28725.9 MB/s)
17#XXH3_stream                   :     102400 ->   243056 it/s (23735.9 MB/s)

compiled as 64-bit x86_64 + AVX2 little endian with GCC 10.2.0
 5#XXH3_64b                      :     102400 ->   288812 it/s (28204.3 MB/s)
17#XXH3_stream                   :     102400 ->   237645 it/s (23207.5 MB/s)

So streaming performance should be "in the same range" as one-shot hashing.

That being said, I've already observed wild performance jumps depending on tiny details, such as compiler versions.
Hence I'm not too surprised by your experience.
This situation is not satisfying, without clearer insight on what's going on behind compiler's magic, it's very difficult to investigate and fix.

I should also try to collect benchmark figures with MSVC, as the reported issue might be specific to this compiler.

I do not observe this impact on my laptop: ``` compiled as 64-bit x86_64 + AVX2 little endian with Apple Clang 11.0.3 (clang-1103.0.32.62) 5#XXH3_64b : 102400 -> 294153 it/s (28725.9 MB/s) 17#XXH3_stream : 102400 -> 243056 it/s (23735.9 MB/s) ``` ``` compiled as 64-bit x86_64 + AVX2 little endian with GCC 10.2.0 5#XXH3_64b : 102400 -> 288812 it/s (28204.3 MB/s) 17#XXH3_stream : 102400 -> 237645 it/s (23207.5 MB/s) ``` So streaming performance should be "in the same range" as one-shot hashing. That being said, I've already observed wild performance jumps depending on tiny details, such as compiler versions. Hence I'm not too surprised by your experience. This situation is not satisfying, without clearer insight on what's going on behind compiler's magic, it's very difficult to investigate and fix. I should also try to collect benchmark figures with MSVC, as the reported issue might be specific to this compiler.

pstadermann commented

2021-03-17 13:29:45 -05:00

(Migrated from github.com)

Thank you for your quick response!

The issue is due to MSVC. I compiled with CLANG under Visual Studio and got these numbers:

compiled as 64-bit x86_64 + AVX2 little endian with Clang 11.0.0
Sample of 100 KB...
5#XXH3_64b : 102400 -> 616253 it/s (60181.0 MB/s)
17#XXH3_stream : 102400 -> 537132 it/s (52454.3 MB/s)

Thank you for your quick response! The issue is due to MSVC. I compiled with CLANG under Visual Studio and got these numbers: compiled as 64-bit x86_64 + AVX2 little endian with Clang 11.0.0 Sample of 100 KB... 5#XXH3_64b : 102400 -> 616253 it/s (60181.0 MB/s) 17#XXH3_stream : 102400 -> 537132 it/s (52454.3 MB/s)

👍 1

pstadermann commented

2021-03-17 13:30:56 -05:00

(Migrated from github.com)

With an updated MSVC 19.28.29913.00 the numbers are equally bad.

Cyan4973 commented

2021-08-10 00:20:14 -05:00

(Migrated from github.com)

I have a new Windows + Visual Studio environment,
under which I could finally test properly this scenario.

I confirm the observed behavior, using a recent compiler :

xxhsum.exe 0.8.1 by Yann Collet
compiled as 64-bit x86_64 + AVX2 little endian with MSVC 19.29.30040.00
Sample of 100 KB...
 5#XXH3_64b                      :     102400 ->   395368 it/s (38610.2 MB/s)
17#XXH3_stream                   :     102400 ->   193160 it/s (18863.3 MB/s)

Indeed, streaming mode introduces a pretty huge hit to performance with MSVC compiler.
This effect is not so pronounced when using the clang backend (still through MSVC IDE) :

xxhsum.exe 0.8.1 by Yann Collet
compiled as 64-bit x86_64 + AVX2 little endian with Clang 11.0.0
Sample of 100 KB...
 5#XXH3_64b                      :     102400 ->   405827 it/s (39631.6 MB/s)
17#XXH3_stream                   :     102400 ->   354967 it/s (34664.7 MB/s)

My guess is that MSVC misses the big in-vector loop when doing streaming, and thus tends to break operations into many smaller batches (for example, at every invocation of XXH3_consumeStripes()), trans-piling data from the registers to memory too often.

It's not obvious how to fix that (nor if that's even possible). The first idea is to ensure that XXH3_consumeStripes() is inlined, so that consecutive calls in the same loop are not broken. But it is. So I'm a bit clueless about possible next step.

On the positive side, I'm actually impressed by MSVC compiler results in one-shot mode, as they are on par with clang. There used to be a much more pronounced difference there.

Note : I note that the problem is especially bad with AVX2.
With SSE2, streaming mode loses performance too, but it's much better contained (-15%)
(By comparison, streaming mode loses no performance with clang on SSE2).
Even with AVX512, the drop of performance is important yet proportionally less severe (-40%) than AVX2 (-52%).

Note 2: On a completely unrelated note, I noticed that performance of XXH32 and XXH64 plummets when xxHash is compiled with MSVC 2019 with AVX2 enabled. Note sure why. Maybe the compiler is trying too hard to vectorize these codes, but they shouldn't be.

I have a new Windows + Visual Studio environment, under which I could finally test properly this scenario. I confirm the observed behavior, using a recent compiler : ``` xxhsum.exe 0.8.1 by Yann Collet compiled as 64-bit x86_64 + AVX2 little endian with MSVC 19.29.30040.00 Sample of 100 KB... 5#XXH3_64b : 102400 -> 395368 it/s (38610.2 MB/s) 17#XXH3_stream : 102400 -> 193160 it/s (18863.3 MB/s) ``` Indeed, streaming mode introduces a pretty huge hit to performance with MSVC compiler. This effect is not so pronounced when using the `clang` backend (still through MSVC IDE) : ``` xxhsum.exe 0.8.1 by Yann Collet compiled as 64-bit x86_64 + AVX2 little endian with Clang 11.0.0 Sample of 100 KB... 5#XXH3_64b : 102400 -> 405827 it/s (39631.6 MB/s) 17#XXH3_stream : 102400 -> 354967 it/s (34664.7 MB/s) ``` My guess is that MSVC misses the big in-vector loop when doing streaming, and thus tends to break operations into many smaller batches (for example, at every invocation of `XXH3_consumeStripes()`), trans-piling data from the registers to memory too often. It's not obvious how to fix that (nor if that's even possible). The first idea is to ensure that `XXH3_consumeStripes()` is inlined, so that consecutive calls in the same loop are not broken. But it is. So I'm a bit clueless about possible next step. On the positive side, I'm actually impressed by MSVC compiler results in one-shot mode, as they are on par with `clang`. There used to be a much more pronounced difference there. _Note_ : I note that the problem is especially bad with `AVX2`. With `SSE2`, streaming mode loses performance too, but it's much better contained (-15%) (By comparison, streaming mode loses no performance with `clang` on `SSE2`). Even with `AVX512`, the drop of performance is important yet proportionally less severe (-40%) than AVX2 (-52%). _Note 2_: On a completely unrelated note, I noticed that performance of `XXH32` and `XXH64` plummets when `xxHash` is compiled with `MSVC 2019` with `AVX2` enabled. Note sure why. Maybe the compiler is trying too hard to vectorize these codes, but they shouldn't be.

Cyan4973 commented

2021-11-27 19:27:16 -06:00

(Migrated from github.com)

Fixed in latest dev branch update

Fixed in latest `dev` branch update

Rows
Columns

Streaming much slower than XXH3_64b for AVX 2 on MSVC #516