Generalize dispatch add-in

julian-klode commented

2020-12-15 07:04:38 -06:00

(Migrated from github.com)

Currently a client needs to look for xxhx86dispatch.h and see if it can build against it, and link to the dispatch symbols, which is a bit unfortunate:

Code written to include xxhash.h will not automatically benefit from optimized implementations
Optimized variants for other architectures would require their own header or the header name to be changed

Generally, from a system library perspective it would be beneficial to have one way to use whatever fastest xxhash there is, whether it's an optimized X86 dispatch thing or a future optimized ARM version, for example.

I think the answer is to add a more generic name xxhash_dispatch.h (xxhash_simd.h?, also keeping the old name around for compatibility), and either install that (DISPATCH=1) or make it just an include for xxhash.h if DISPATCH=0, such that you can always just #include <xxhash_dispatch.h> and you'll get whatever fastest version is available; and then you have xxhash.h left in case you want a non-dispatched version, which maybe you have valid reasons for (short data sizes?).

The other alternative would be to make xxhash.h be the dispatched one if DISPATCH=1, and install real "xxhash.h" as xxhash_basic.h, such that everyone using xxhash.h gets dispatched versions automatically.

Currently a client needs to look for xxhx86dispatch.h and see if it can build against it, and link to the dispatch symbols, which is a bit unfortunate: - Code written to include xxhash.h will not automatically benefit from optimized implementations - Optimized variants for other architectures would require their own header or the header name to be changed Generally, from a system library perspective it would be beneficial to have one way to use whatever fastest xxhash there is, whether it's an optimized X86 dispatch thing or a future optimized ARM version, for example. I think the answer is to add a more generic name xxhash_dispatch.h (xxhash_simd.h?, also keeping the old name around for compatibility), and either install that (DISPATCH=1) or make it just an include for xxhash.h if DISPATCH=0, such that you can always just `#include <xxhash_dispatch.h>` and you'll get whatever fastest version is available; and then you have xxhash.h left in case you want a non-dispatched version, which maybe you have valid reasons for (short data sizes?). The other alternative would be to make `xxhash.h` be the dispatched one if DISPATCH=1, and install real "xxhash.h" as xxhash_basic.h, such that everyone using xxhash.h gets dispatched versions automatically.

👍 1

julian-klode commented

2020-12-15 07:14:37 -06:00

(Migrated from github.com)

Arguably it's worth wondering if it shouldn't do DISPATCH=1 automatically when building as a shared library. Basically avoiding the need for distributions to set those flags based on architectures, and having to adjust that if an ARM version is added for example.

easyaspi314 commented

2020-12-15 07:41:43 -06:00

(Migrated from github.com)

There is unfortunately no easy way to check if ARMv7 has NEON.

The architecture has it (it is in the HCPTR register), but it can't be accessed from user code. The best option is to use platform-specific things like /proc/cpuinfo which is very ugly.

There is unfortunately no easy way to check if ARMv7 has NEON. The architecture has it (it is in the HCPTR register), but it can't be accessed from user code. The best option is to use platform-specific things like `/proc/cpuinfo` which is very ugly.

julian-klode commented

2020-12-15 09:18:57 -06:00

(Migrated from github.com)

The answer on Linux would be getauxval(AT_HWCAP) & HWCAP_NEON is my understanding. And yes, it will get platform specific, but oh well, that's a price you gotta pay for optimal performance - and it's only in a dispatch.c file, so meh.

The answer on Linux would be `getauxval(AT_HWCAP) & HWCAP_NEON` is my understanding. And yes, it will get platform specific, but oh well, that's a price you gotta pay for optimal performance - and it's only in a dispatch.c file, so meh.

Cyan4973 commented

2023-07-15 15:38:08 -05:00

(Migrated from github.com)

This objective has been set for v0.9.0

This objective has been set for `v0.9.0`

Cyan4973 commented

2024-12-26 20:10:58 -06:00

(Migrated from github.com)

Next version of xxhsum will automatically compile with DISPATCH=1 when target is detected as x86/x64.

Note that this DISPATCH=1 functionality is for xxhsum, the command line interface, and not for libxxhash, the library.

Next version of `xxhsum` will automatically compile with `DISPATCH=1` when target is detected as `x86`/`x64`. Note that this `DISPATCH=1` functionality is for `xxhsum`, the command line interface, and not for `libxxhash`, the library.

Rows
Columns

Generalize dispatch add-in #483