-
xxHash v0.8.4 Stable
released this
2026-09-18 19:29:23 -05:00 | -10 commits to dev since this releasexxHash
v0.8.4is a general maintenance update, offering broader hardware acceleration, command line enhancements, and more predictable builds across platforms. No change to the API entry points.Faster performance with vectorization
This release adds several vectorized implementations: for the
RISC-VVector extension and LoongArch LASX, together with improvements to the existing LSX implementation.On
aarch64,XXH3now prefersNEONoverSVE. On tested processors, this improves performance by 25–40%.XXH3streaming performance has also been improved for workloads made of small updates.Command-line enhancements
xxhsumcan now receive a full 64-bit seed through-sor--seed.Command-line parsing is more flexible: options may appear anywhere, even after filenames, while
--explicitly terminates option processing.When processing a list of files, read errors no longer prevent subsequent files from being processed.
Windows support received a substantial update, notably for long paths, UNC paths and filenames containing components that require extended
Win32path handling.Build and integration
Runtime dispatch detection has been corrected for cross-compilation and non-x86 targets.
makebuilds now isolate object files by configuration, avoiding accidental reuse across incompatible flag sets and allowing parallel test builds.The
cmakeintegration moved fromcmake_unofficialtobuild/cmake. Projects referring directly to the previous directory must update that path.Low-level integrations can now redirect
XXH_memcpy(),XXH_memset()andXXH_memcmp(). A newXXH_NO_EXTERNC_GUARDmacro is also available for projects that need to manage C++ linkage themselves.Correctness and portability
A strict-aliasing issue that could cause
gccto miscompile some unaligned reads has been fixed. This release also addresses combinations involvingXXH_INLINE_ALL, namespaces and x86 runtime dispatch, system headers included from anextern "C"block, and MSVC link-time optimization with warnings enabled as errors.Portability improvements include compatibility with ARM builds using GCC versions older than 8, C++ compilation on Solaris and illumos, and improved installation metadata and paths on Windows and FreeBSD.
Detailed changelist
- perf : added RISC-V Vector extension (RVV) implementation, by @zijianli1234, @BoBoDai and @camel-cdr
- perf : added LoongArch LASX implementation and improved LSX, by @24bit-xjkp
- perf : prefer NEON over SVE on AArch64, improving XXH3 speed by 25–40%, by @Nicoshev
- perf : improved XXH3 streaming speed for very small updates, suggested by @aleksazr
- api :
XXH_memcpy(),XXH_memset()andXXH_memcmp()can be redirected at compile time - api : new
XXH_NO_EXTERNC_GUARDbuild macro for complex C++ integration - cli : added 64-bit seed support (
-s,--seed), by @d06i - cli : options can appear after filenames;
--terminates option parsing, #1098 reported by @unterkomplex - cli : continue processing subsequent files after read errors, #1064 reported by @ZoomRmc
- cli : support long and special Windows paths, #1060, based on #1061 by @t-mat
- cli : improved benchmark accuracy, error handling and unit labels, #1051 reported by @calgray
- fix : unaligned reads could be miscompiled by GCC under strict aliasing, #1013 by @lassipulkkinen
- fix :
XXH_INLINE_ALLcompatibility withXXH_NAMESPACEand x86 runtime dispatch, by @pps83, #1071 reported by @Dr-Hex - fix : avoid including system headers inside an
extern "C"block, #1122 reported by @andreas-schwab - fix : compilation with MSVC link-time optimization and warnings as errors, #1038 reported by @mi2think
- build: improved runtime dispatch detection for cross-compilation and non-x86 targets, with help from @eworm-de, #1081 reported by @PotMJ
- build:
LIBXXH_DISPATCH=1now applies to static and dynamic libraries, #1058 reported by @xiota - build: Make builds now isolate objects by configuration and support parallel test builds
- build: moved CMake integration to
build/cmakeand improved cross-compilation detection - install: improved package metadata and installation paths on Windows and FreeBSD, by @iwamatsu, @chitao1234 and @sunpoet
- portability: fixed compilation on ARM with GCC < 8, by @mstorsjo
- portability: fixed C++ compilation on Solaris and illumos, by @luqmana
- doc : expanded
xxhsumdocumentation and clarified the algorithm specification files - doc : documented non-cryptographic collision properties, notably for XXH3's mid-size path, #1127 by @thomasahle
Full changelog: https://github.com/Cyan4973/xxHash/compare/v0.8.3...v0.8.4
New Contributors
- @sunpoet made their first contribution in https://github.com/Cyan4973/xxHash/pull/994
- @24bit-xjkp made their first contribution in https://github.com/Cyan4973/xxHash/pull/996
- @walterdejong made their first contribution in https://github.com/Cyan4973/xxHash/pull/993
- @chitao1234 made their first contribution in https://github.com/Cyan4973/xxHash/pull/1008
- @Schell-HAR made their first contribution in https://github.com/Cyan4973/xxHash/pull/1009
- @musvaage made their first contribution in https://github.com/Cyan4973/xxHash/pull/1021
- @iwamatsu made their first contribution in https://github.com/Cyan4973/xxHash/pull/1024
- @Nicoshev made their first contribution in https://github.com/Cyan4973/xxHash/pull/1023
- @zijianli1234 made their first contribution in https://github.com/Cyan4973/xxHash/pull/1043
- @lassipulkkinen made their first contribution in https://github.com/Cyan4973/xxHash/pull/1013
- @YexuanXiao made their first contribution in https://github.com/Cyan4973/xxHash/pull/1052
- @eworm-de made their first contribution in https://github.com/Cyan4973/xxHash/pull/1055
- @stefan-sedlak made their first contribution in https://github.com/Cyan4973/xxHash/pull/1068
- @BoBoDai made their first contribution in https://github.com/Cyan4973/xxHash/pull/1069
- @camel-cdr made their first contribution in https://github.com/Cyan4973/xxHash/pull/1066
- @FatihBAKIR made their first contribution in https://github.com/Cyan4973/xxHash/pull/1073
- @luqmana made their first contribution in https://github.com/Cyan4973/xxHash/pull/1093
- @marcusmueller made their first contribution in https://github.com/Cyan4973/xxHash/pull/1096
- @mstorsjo made their first contribution in https://github.com/Cyan4973/xxHash/pull/1092
- @d06i made their first contribution in https://github.com/Cyan4973/xxHash/pull/1088
- @NexusXe made their first contribution in https://github.com/Cyan4973/xxHash/pull/1101
Downloads
-
Source code (ZIP)
0 downloads
-
Source code (TAR.GZ)
0 downloads
-
xxHash v0.8.3 Stable
released this
2024-12-29 14:58:19 -06:00 | 277 commits to dev since this releasexxHash
v0.8.3is a maintenance update, featuring a bug fix and several quality of life improvements.Bug Fix
XXH3_128bits_withSecretandSeed()
Corrects an edge case (#894) that could generate invalid results. Users of this function should upgrade. Thanks to @hltj for the report and fix.
Command-Line Improvements
- Runtime Vector Extension on x86
xxhsumautomatically detects and employs the best available vector extension (SSE, AVX, etc.) onx86/x64cpus. Previously, this required an explicit build macro; it is now enabled by default. Maintainers can still control it manually, and can disable it withDISPATCH=0. --filelist/--files-from
Accepts file names from a text file orstdin, simplifying bulk hashing. Kudos to @Ian-Clowes for the idea and implementation.- Short GNU Format for XXH3
A new-H3mode generates XXH3 64-bit hashes with aXXH3_prefix, neatly distinguishing them fromXXH64. The symlinkxxh3sumdefaults to this mode.
Portability
- LoongArch SX SIMD Support
Includes an optimized LoongArch SX implementation of xxh3 (courtesy of @lrzlin). - Extended Platform Coverage
Validated builds for AIX OS and SPARC CPUs.
New Contributors
- @scribam made their first contribution in https://github.com/Cyan4973/xxHash/pull/859
- @klemensn made their first contribution in https://github.com/Cyan4973/xxHash/pull/880
- @thestr4ng3r made their first contribution in https://github.com/Cyan4973/xxHash/pull/878
- @cclauss made their first contribution in https://github.com/Cyan4973/xxHash/pull/895
- @nathaniel-brough made their first contribution in https://github.com/Cyan4973/xxHash/pull/906
- @crosdahl made their first contribution in https://github.com/Cyan4973/xxHash/pull/932
- @pps83 made their first contribution in https://github.com/Cyan4973/xxHash/pull/931
- @hltj made their first contribution in https://github.com/Cyan4973/xxHash/pull/894
- @spaette made their first contribution in https://github.com/Cyan4973/xxHash/pull/953
- @mofosyne made their first contribution in https://github.com/Cyan4973/xxHash/pull/954
- @Ian-Clowes made their first contribution in https://github.com/Cyan4973/xxHash/pull/972
- @crrodriguez made their first contribution in https://github.com/Cyan4973/xxHash/pull/976
- @lrzlin made their first contribution in https://github.com/Cyan4973/xxHash/pull/981
Full Changelog: https://github.com/Cyan4973/xxHash/compare/v0.8.2...v0.8.3
Downloads
-
Source code (ZIP)
0 downloads
-
Source code (TAR.GZ)
0 downloads
-
xxHash v0.8.2 Stable
released this
2023-07-21 05:01:35 -05:00 | 473 commits to dev since this releasexxHash
v0.8.2is an incremental update featuring multiple small improvements and fixes spread out over ~300 commits.Faster performance
Several updates by @easyaspi314 and @hzhuang1 impact
armplatform, most notably theneoncode path. On theM1 Pro, this translates into +20% speed forxxh3andxxh128(from 30.0 GB/s to 36 GB/s).
Some of the changes are generic, so other platforms can be affected too, though typically to a lesser extend (~5%).On
wasm, speed foxxh3is improved by a large factor x2 to x3 (depending on underlying hardware) through the use ofsimd128(@easyaspi314). This is especially efficient under thev8js engine, notably used bychromeandnode.js.Finally, @hzhuang1 added support for the
arm'sSVEvector extension. This is useful for server-sideaarch64cpus with hardware support for wide vectors, such as Fujitsu's A64FX.Fixes and improvements
Notable fixes in this update include the resolution of issues with
XXH3S390xvector implementation,PowerPCvector compilation with IBM XL compiler, and-Ogcompilation.Furthermore, the command line interface (CLI) was refined with features such as support for comment lines in check files and commands such as
--binaryand--ignore-missing(@t-mat). Additionally, issues with filename containing/LFcharacter were resolved.The build process was also refined, with improvements such as fixing
pkgconfiggeneration withcmake(@ilya-fedin),icccompilation,cmakeinstall directories, and new build options to reduce binary size (@easyaspi314). Dedicated install targets were introduced (@ffontaine), and support forDISPATCHmode in cmake was added (@hzhuang1).In terms of portability, the update includes the
SVEvector implementation ofXXH3, compatibility with freestanding environments usingXXH_NO_STDLIB, and the ability to build on Haiku. The code has also been validated onm68kandrisc-v.Documentation
XXH3finally has a written specification, thanks to @Adrien1018 !
Source code can also be digested bydoxygento generate code documentation automatically. An instance is now available at homepage.Erratum
There is a bug in this version when invoking the function
XXH3_128bits_withSecretandSeed(), specifically when the parameterseed == 0, and inputlength < XXH3_MIDSIZE_MAX(< 240 bytes), and thesecretis different from the one created withXXH3_generateSecret_fromSeed(), and the user is invoking the Streaming API. The hash values produced in this case are incorrect: as stated in the documentation, they should be== XXH3_128bits_withSeed(). This is fixed in later version and thedevbranch , thanks to @hltj.(a9b2f1811534c5cbb8a7a702192c919ac320e4bb).Changelog
- fix : XXH3
S390xvector implementation (@hzhuang1) - fix : PowerPC vector compilation with IBM XL compiler (@MaxiBoether)
- perf : improved
WASMspeed by x2/x3 usingSIMD128(@easyaspi314) - perf : improved speed (+20%) for XXH3 on ARM
NEON(@easyaspi314) - cli : Fix filename contain
/LFcharacter (@t-mat) - cli : Support
#comment lines in--checkfiles (@t-mat) - cli : Support commands
--binaryand--ignore-missing(@t-mat) - build: fix
-Ogcompilation (@easyaspi314, @t-mat) - build: fix
pkgconfiggeneration withcmake(@ilya-fedin) - build: fix
icccompilation - build: fix
cmakeinstall directories - build: new build options
XXH_NO_XXH3,XXH_SIZE_OPTandXXH_NO_STREAMto reduce binary size (@easyaspi314) - build: dedicated install targets (@ffontaine)
- build: support
DISPATCHmode incmake(@hzhuang1) - portability: fix
x86dispatchwhen building withVisual+ clang-cl (@t-mat) - portability:
SVEvector implementation of XXH3 (@hzhuang1) - portability: compatibility with freestanding environments, using
XXH_NO_STDLIB - portability: can build on Haiku (@Begasus)
- portability: validated on
m68kandrisc-v - doc : XXH3 specification (@Adrien1018)
- doc : improved doxygen documentation (@easyaspi314, @t-mat)
- misc : dedicated sanity test binary (@t-mat)
Full change list (github generated)
- Fix an assert comparison the same values (flagged by PVS Studio in 0.8.1) by @kcgen in https://github.com/Cyan4973/xxHash/pull/628
- Add GitHub Actions badge for release branch by @t-mat in https://github.com/Cyan4973/xxHash/pull/633
- Add windows-2022 to ci.yml by @t-mat in https://github.com/Cyan4973/xxHash/pull/634
- Add macOS matrix to ci.yml by @t-mat in https://github.com/Cyan4973/xxHash/pull/635
- Fix compilation on RHEL 7 ppc64le (gcc 4.8) by @ellert in https://github.com/Cyan4973/xxHash/pull/631
- Add clang-cl for MSVC 2019 to ci.yml by @t-mat in https://github.com/Cyan4973/xxHash/pull/637
- [NEON] Split XXH3 into 6 NEON lanes and 2 scalar lanes on aarch64 by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/632
- Fix some ARM/clang-cl feature detection issues by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/623
- Add QEMU/gcc matrix to ci.yml by @t-mat in https://github.com/Cyan4973/xxHash/pull/640
- fix #625 by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/638
- fix #627 by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/639
- added m68k emulation tests to GA by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/643
- Document some nerdy ARM stuff, move scalarRound down. by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/642
- fix minor static analyzer warning by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/644
- fix man page installation by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/648
- fix cmake --install by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/649
- Use attribute((aligned)) instead of packed by @Hello71 in https://github.com/Cyan4973/xxHash/pull/650
- [ARM/AArch64] Fix multiple GCC codegen problems by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/651
- removed XXH3 declarations when XXH_NO_XXH3 is defined by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/653
- new build macro XXH_NO_STDLIB by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/654
- improved nostdlib test by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/656
- added attribute((const)) by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/657
- added attribute((malloc)) by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/658
- added attribute((pure)) by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/659
- Documentation update by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/661
- Makefile: add dedicated install targets by @ffontaine in https://github.com/Cyan4973/xxHash/pull/665
- XXH_HAS_C_ATTRIBUTE(x)?! by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/662
- do no longer depend on
<assert.h>for XXH_STATIC_ASSERT by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/670 - Properly fix altivec namespace collisions by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/672
- Introduce XXH_SIZE_OPT and XXH_NO_STREAM by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/667
- Remove duplicated definition of XXH3_128bits by @mterron in https://github.com/Cyan4973/xxHash/pull/676
- Removed windows-2016 from ci.yml by @t-mat in https://github.com/Cyan4973/xxHash/pull/690
- tipi.build instructions by @pysco68 in https://github.com/Cyan4973/xxHash/pull/688
- Fix issue #695 by @t-mat in https://github.com/Cyan4973/xxHash/pull/698
- Build fix for Haiku by @Begasus in https://github.com/Cyan4973/xxHash/pull/696
- Use inline assembler for Power/IBM XL Compiler by @MaxiBoether in https://github.com/Cyan4973/xxHash/pull/708
- test filename-escape by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/710
- avoid add_compile_definitions for cmake < v3.12 by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/711
- just more cmake v2.8.12 tests by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/721
- CPack Added in https://github.com/Cyan4973/xxHash/pull/719
- Remove stream loads and slightly improve avx512 seed generation by @goldsteinn in https://github.com/Cyan4973/xxHash/pull/726
- Fix: brace expansion by @t-mat in https://github.com/Cyan4973/xxHash/pull/729
- Fix issue #724 by @t-mat in https://github.com/Cyan4973/xxHash/pull/730
- Remove macOS-10.15 from ci.yml by @t-mat in https://github.com/Cyan4973/xxHash/pull/736
- blind fix for fallthrough on icc by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/718
- Optimize XXH3_accumulate_512_neon by @dougallj in https://github.com/Cyan4973/xxHash/pull/734
- Fix typos found by codespell by @DimitriPapadopoulos in https://github.com/Cyan4973/xxHash/pull/739
- ci: fix tipi build error on github CI workflow by @hzhuang1 in https://github.com/Cyan4973/xxHash/pull/749
- Update GitHub Actions by @DimitriPapadopoulos in https://github.com/Cyan4973/xxHash/pull/742
- xxhash: support SVE by intrinsic code by @hzhuang1 in https://github.com/Cyan4973/xxHash/pull/752
- fix issues reported by cppcheck by @hzhuang1 in https://github.com/Cyan4973/xxHash/pull/746
- CI: fix missing space by @hzhuang1 in https://github.com/Cyan4973/xxHash/pull/758
- Fixing tipi-build / Build as dependency CI step by @pysco68 in https://github.com/Cyan4973/xxHash/pull/760
- Customize full accumulating loop for SVE by @hzhuang1 in https://github.com/Cyan4973/xxHash/pull/756
- added macos-12 test to GH CI by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/765
- Small improvement to x86 vectorized hashes and medium-sizes hash. by @goldsteinn in https://github.com/Cyan4973/xxHash/pull/754
- dispatch: Use attribute((constructor)) on XXH_setDispatch by @goldsteinn in https://github.com/Cyan4973/xxHash/pull/773
- Fix typo found by codespell by @DimitriPapadopoulos in https://github.com/Cyan4973/xxHash/pull/777
- http:// → https:// by @DimitriPapadopoulos in https://github.com/Cyan4973/xxHash/pull/778
- Add attribute((noescape)) on pointers in public API by @goldsteinn in https://github.com/Cyan4973/xxHash/pull/782
- Annotate xxh_x86dispatch with XXH_NOESCAPE by @goldsteinn in https://github.com/Cyan4973/xxHash/pull/786
- [NEON] Refactor NEON code by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/787
- Makefile: add install_libxxhash.includes target by @ffontaine in https://github.com/Cyan4973/xxHash/pull/776
- s390x: add memory barrier to avoid fault issue by @hzhuang1 in https://github.com/Cyan4973/xxHash/pull/785
- Define XXH_RESTRICT for some C++ compilers (gcc/clang/msvc/icc) by @goldsteinn in https://github.com/Cyan4973/xxHash/pull/788
- Fix spelling. by @dwsteele in https://github.com/Cyan4973/xxHash/pull/797
- cmake: support dispatch mode by @hzhuang1 in https://github.com/Cyan4973/xxHash/pull/790
- Fix
XXH_UNREACHABLEfeature test for C++23 and remove for C23 #791 by @goldsteinn in https://github.com/Cyan4973/xxHash/pull/792 - Rewrite XXH3_update by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/805
- fix: GH-Actions - remove ubuntu-18.04 by @t-mat in https://github.com/Cyan4973/xxHash/pull/817
- Fix: add gcc -Og check and fix its false positive warnings by @t-mat in https://github.com/Cyan4973/xxHash/pull/814
- Fix off-by-one in XXH3_consumeStripes() (Fixes #816) by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/818
- added read-all permission status to Github Actions Workflow by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/807
- XXH_ASSUME macro using
__builtin_assumeif supported (clang only fo… by @devnexen in https://github.com/Cyan4973/xxHash/pull/803 - Fix GCC 12 -Og by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/804
- Add
gcc-12to ci.yml by @t-mat in https://github.com/Cyan4973/xxHash/pull/819 - benchmark : adjust initial speed target to 10 MB/s by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/822
- added one test length for xxh sanity checks by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/820
- Various AArch64 speed hacks by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/809
- Dedicated sanity test program by @t-mat in https://github.com/Cyan4973/xxHash/pull/823
- Add Security Policy by @gabibguti in https://github.com/Cyan4973/xxHash/pull/828
- Reference actions by commit SHA by @gabibguti in https://github.com/Cyan4973/xxHash/pull/833
- Bump actions/checkout from 3.5.2 to 3.5.3 by @dependabot in https://github.com/Cyan4973/xxHash/pull/837
- removed tipi test from CI suite because it's unstable by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/841
- reworded the AVX error message when compiling xxh_x86dispatch.c by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/840
- Feature: Support # comment lines in --check files by @t-mat in https://github.com/Cyan4973/xxHash/pull/838
- Fix double prefix when building with cmake by @ilya-fedin in https://github.com/Cyan4973/xxHash/pull/723
- added a cmake pkgconfig generation test by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/845
- added RISC-V emulation to CI by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/846
- Add algorithm specification of XXH3 by @adrien1018 in https://github.com/Cyan4973/xxHash/pull/750
- minor follow-ups for XXH3 specification (#750) by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/847
- Remove
0.8.1s by @t-mat in https://github.com/Cyan4973/xxHash/pull/848 - Detect and display SVE when active by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/849
- Fix doxygen warnings (2nd attempt) by @t-mat in https://github.com/Cyan4973/xxHash/pull/851
- feature: CLI: --binary by @t-mat in https://github.com/Cyan4973/xxHash/pull/852
- Add WebAssembly SIMD128 implementation and Node.JS support by @easyaspi314 in https://github.com/Cyan4973/xxHash/pull/825
- Add --ignore-missing by @t-mat in https://github.com/Cyan4973/xxHash/pull/853
- Add missing doxygen style comments to XXH64 family by @t-mat in https://github.com/Cyan4973/xxHash/pull/855
- Add missing doxygen style comments to XXH3 family by @t-mat in https://github.com/Cyan4973/xxHash/pull/856
- Add --ignore-missing and --binary to --help and xxhsum.1 by @t-mat in https://github.com/Cyan4973/xxHash/pull/857
- added a compilation warning message for XXH_OLD_NAMES by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/860
- fix: __cpuidex pointer type warning by @t-mat in https://github.com/Cyan4973/xxHash/pull/864
- Fix clang-cl dispatch by @t-mat in https://github.com/Cyan4973/xxHash/pull/865
- add a test combining Windows + Visual + Clang-Cl + DISPATCH=1 by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/863
- fix: enable clang-cl test for VC++2022 by @t-mat in https://github.com/Cyan4973/xxHash/pull/866
- fix: Intel compiler's warning for XXH_ASSERT() by @t-mat in https://github.com/Cyan4973/xxHash/pull/867
- fix: hide internal macros from doxygen by @t-mat in https://github.com/Cyan4973/xxHash/pull/868
- hide x86 dispatcher internals by @t-mat in https://github.com/Cyan4973/xxHash/pull/869
- CI: Add gcc-13 and clang-{13,14,15} by @t-mat in https://github.com/Cyan4973/xxHash/pull/872
- revert XXH3's 17to128 variant to single accumulator by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/871
- v0.8.2 by @Cyan4973 in https://github.com/Cyan4973/xxHash/pull/873
New Contributors
- @kcgen made their first contribution in https://github.com/Cyan4973/xxHash/pull/628
- @Hello71 made their first contribution in https://github.com/Cyan4973/xxHash/pull/650
- @ffontaine made their first contribution in https://github.com/Cyan4973/xxHash/pull/665
- @mterron made their first contribution in https://github.com/Cyan4973/xxHash/pull/676
- @pysco68 made their first contribution in https://github.com/Cyan4973/xxHash/pull/688
- @MaxiBoether made their first contribution in https://github.com/Cyan4973/xxHash/pull/708
- @goldsteinn made their first contribution in https://github.com/Cyan4973/xxHash/pull/726
- @dougallj made their first contribution in https://github.com/Cyan4973/xxHash/pull/734
- @DimitriPapadopoulos made their first contribution in https://github.com/Cyan4973/xxHash/pull/739
- @hzhuang1 made their first contribution in https://github.com/Cyan4973/xxHash/pull/749
- @dwsteele made their first contribution in https://github.com/Cyan4973/xxHash/pull/797
- @devnexen made their first contribution in https://github.com/Cyan4973/xxHash/pull/803
- @gabibguti made their first contribution in https://github.com/Cyan4973/xxHash/pull/828
- @dependabot made their first contribution in https://github.com/Cyan4973/xxHash/pull/837
- @ilya-fedin made their first contribution in https://github.com/Cyan4973/xxHash/pull/723
- @adrien1018 made their first contribution in https://github.com/Cyan4973/xxHash/pull/750
Full Changelog: https://github.com/Cyan4973/xxHash/compare/v0.8.1...v0.8.2
Downloads
-
Source code (ZIP)
0 downloads
-
Source code (TAR.GZ)
0 downloads
- fix : XXH3
-
v0.8.1 Stable
released this
2021-11-29 12:34:10 -06:00 | 785 commits to dev since this releasexxHash
v0.8.1is a general clean up of the code base, following the stabilization ofxxh3andxxh128inv0.8.0.
There are a few welcomed evolutions and improvements, but for the most part, this release consists of fixes for multiple corner cases and scenarios, that shall improve usability oflibxxhashandxxhsumacross a wide range of platforms.
StableAPIentry points have not changed, all entry points labelled "stable" will continue to work as intended in this release and future ones.Improved performance
While the "big picture" is unchanged, there are a few notable improvements.
XXH3/XXH128feature a large speed improvement in streaming mode, which is particularly sensible forgccandMSVC(clangwas already in good shape), by as much as +40%, making streaming speed essentially on par with single-shot mode when ingesting large quantities of data.XXH64and evenXXH32feature improved latency performance for small inputs of random sizes. Perhaps as importantly, their binary size is smaller.New capabilities
There is a new experimental
XXH3variant, named_withSecretandSeed(). In a nutshell, it combinesseedfor small inputs, withsecretfor large inputs.
The main driver for this variant is a wish to skip the delay fromsecret's transparent generation when using_withSeed()variant with large inputs, resulting in measurable performance drop for "not so large" sizes (< 1 KB) (note: this delay is insensible for "large" inputs, such as > 256 KB). Coupled with new functionXXH3_generateSecret_fromSeed(), which generates the same secret as the one generated internally when using the_withSeed()variant, it results in exactly the same return values, while skipping thesecretgeneration stage, thus improving speed.Experimental
XXH3_generateSecret()has been extended to allow generation ofsecretof any size (though respecting the specification's minimum size). It's generally recommended to use this generator to ensure a source of "high entropy" for thesecret.On the CLI front, a highly demanded
xxhsumfeature was an ability to generateXXH3checksum values. This is achieved inv0.8.1, using the--tagformat, which ensures thatXXH3results cannot be confused with (default)XXH64ones, even though they feature the same 64-bit width.Detailed changelist
- perf : much improved performance for
XXH3streaming variants, notably ongccandmsvc - perf : improved
XXH64speed and latency on small inputs - perf : small
XXH32speed and latency improvement on small inputs of random size - perf : minor stack usage improvement for
XXH32andXXH64 - api : new experimental variants
XXH3_*_withSecretandSeed() - api : updated
XXH3_generateSecret(), can now generate secret of any size (>= XXH3_SECRET_SIZE_MIN) - cli :
xxhsumcan now generate and checkXXH3checksums, using command-H3 - build: can build xxhash without
XXH3, with new build macroXXH_NO_XXH3 - build: fix
xxh_x86dispatchbuild with MSVC, by @apankrat - build:
XXH_INLINE_ALLcan always be used safely, even afterXXH_NAMESPACEor a previousXXH_INLINE_ALL - build: improved PPC64LE vector support, by @mpe
- install: fix pkgconfig, by @ellert
- install: compatibility with Haiku, by @Begasus
- doc : code comments made compatible with doxygen, by @easyaspi314
- misc :
XXH_ACCEPT_NULL_INPUT_POINTERis no longer necessary, all functions can acceptNULLinput pointers, as long assize == 0 - misc : complete refactor of CI tests on Github Actions, offering much larger coverage, by @t-mat
- misc :
xxhsumcode base split into multiple specialized units, within directorycli/, by @easyaspi314
Downloads
-
Source code (ZIP)
0 downloads
-
Source code (TAR.GZ)
0 downloads
- perf : much improved performance for
-
xxHash v0.8.0 - Stable XXH3 Stable
released this
2020-07-27 13:57:06 -05:00 | 1057 commits to dev since this releaseStable XXH3
After more than a year in the making,
XXH3has finally reachedstablestatus, for both its 64-bit and 128-bit variants.
While the code itself was in good enough shape for production use, the generated values could still change between versions. This limitedXXH3to local sessions only.
From now on, output values produced byXXH3for a given input and parameter set will remain identical across systems and across future versions. It makes it possible to store these values for later comparison, or to exchange them across network connections.BSD-style checksums
Official stabilization being the main goal of this release, there are only minimal additional changes.
A notable one though is the ability forxxhsumCLI to produce and check BSD-style checksum lines, using command--tag.
One advantage of--tagformat is that it explicitly specifies the algorithm and format used to represent the checksum. For example, it explicitly mentions if a checksum value follows the canonical format (XXH32) or the alternative little-endian format (XXH32_LE).
Generating BSD-style checksum lines was actually already possible, but as the CLI was unable to--checkthem, it remained a hidden option.
This situation changes withv0.8.0, thanks to a patch by @WayneD which makes it possible to--checkBSD-style checksum lines.Detailed list
- api : stabilize XXH3
- cli :
xxhsumcan produce BSD-style lines, with command--tag - cli :
xxhsumcan parse and check BSD-style lines, using command--check, by @WayneD - cli :
xxhsum -accepts console input, requested by @jaki - cli :
xxhsumaccepts--separator, by @jaki - cli : fix : print correct default algo for symlinked helpers, by @martinetd
- install: improved pkgconfig script, allowing custom install locations, requested by @ellert
Downloads
-
Source code (ZIP)
0 downloads
-
Source code (TAR.GZ)
0 downloads
-
released this
2020-06-25 01:32:16 -05:00 | 1128 commits to dev since this releasexxHash
v0.7.4is the last evolution ofxxh3andxxh128, primarily designed to finalize the algorithm.
It is considered release candidate forv0.8.0, which means that if all goes right, this version will rebrandedv0.8.0, almost "as is", within the next few weeks, after receiving sufficient feedback.
v0.8.0is the official version after whichXXH3andXXH128are considered "stabilized", meaning that return values will never change given the sameinputandseed, making the hash suitable for long-term storage and transmission.Beyond these "final touches", the new version also brings a few notable improvements.
Automatic vector detection
x86/x64systems can enjoy a new unit,xxh_x86dispatch, which can detect at runtime the best vector instruction set present on host system (none,sse2,avx2oravx512), thanks to a cpu feature detector designed by @easyaspi314. It then automatically runs the appropriate vector code.
This makes it safer to deploy a single binary with advanced vector instruction sets, such asAVX2, since there is no hard requirement for all target systems to actually support it : the binary can automatically switch toSSE2instead.
As a proof of concept, the windows builds provided alongside this release are compiled with this new capability.AVX512supportA new vector instruction set is supported, thanks to @gzm55 :
AVX512. It can be applied onXXH3andXXH128, using some of the most recent Intel cpus, such as IceLake on laptop. It typically offers +50% more performance compared toAVX2.Secret Generator
Advanced users can be interested in the highly customizable variant
_withSecret(), which makes it possible to runXXH3andXXH128algorithms using one's ownsecret.
However, the quality of the hash depends on the high entropy (randomness) of thesecret. And sometimes, it can be difficult to ensure that the candidatesecretis "random enough".
In order to produce asecretof high quality, a new functionXXH3_generateSecret()is proposed in the advanced API section. It will convert any blob of bytes, namedcustomSeed, into a high qualitysecretwhich respects all conditions expected byXXH3andXXH128. This is true even ifcustomSeeditself is of poor quality, such as a bunch of\0bytes or some short or repeated common sequence.No API modification
The existing API present in 0.7.3 has remained unchanged in 0.7.4. Any programs linking with 0.7.3 should continue to work as-is.
Note however thatxxh3/xxh128return values are not comparable across these versions.
0.7.x are labelled development versions, and should only be used for ephemeral data (hash produced and consume in the same local session).
(note : this limitation does not extend toXXH32andXXH64, which are considered fully stable and specified).Changelist
There are multiple smaller bug fixes and minor improvements that have been brought to this repository by great contributors. Here is a summarized list:
- perf: automatic vector detection and selection at runtime (
xxh_x86dispatch.h), initiated by @easyaspi314 - perf: added
AVX512support, by @gzm55 - api : new: secret generator
XXH_generateSecret(), suggested by @koraa - api : fix:
XXH3_state_tis movable, identified by @koraa - api : fix: state is correctly aligned in AVX mode (unlike
malloc()), by @easyaspi314 - api : fix: streaming generated wrong values in some combination of random ingestion lengths, reported by @WayneD
- cli : fix unicode print on Windows, by @easyaspi314
- cli : can
-ccheck file generated bysfv - build:
make DISPATCH=1generatesxxhsumandlibxxhashwith runtime vector detection (x86/x64 only) - install: cygwin installation support
- doc : Cryptol specification of
XXH32andXXH64, by @weaversa
Downloads
-
Source code (ZIP)
0 downloads
-
Source code (TAR.GZ)
0 downloads
- perf: automatic vector detection and selection at runtime (
-
xxHash v0.7.3 Stable
released this
2020-03-04 20:37:01 -06:00 | 1286 commits to dev since this releasexxHash v0.7.3 is major evolution for
xxh3andxxh128, with a focus on speed and dispersion performance.Speed improvements
v0.7.3pays a lot of attention to small data, by delivering generally faster latency metrics (about +10%).Inlining is now a first class citizen, as it is generally key to best performance on small inputs.
Among the visible changes:XXH_INLINE_ALLcan always be set before includingxxhash.h, even ifxxhash.hwas previously included (for example transitively, as part of a prior*.hheader file).- The algorithm implementation has been transferred into
xxhash.h. It's no longer necessary to keep a copy ofxxhash.cin the/includedirectory for inlining to work correctly.- Note:
xxhash.cstill exists, as it's useful to instantiate xxhash functions as public symbols accessible from a library or a*.oobject file. It also remains compatible with existing projects.
- Note:
Large data has also received a boost, which can go up to +20% for very large samples (> many MB).
Let's underline the remarkable optimization work of @easyaspi314, who hand optimized several hot loops and instructions, and even added a new Z-vector target for
s390xhardware.No API modification
The API has remained completely stable between 0.7.2 and 0.7.3. Any programs linking with 0.7.2 should work as-is.
Note thatxxh3/xxh128results are not comparable across these versions.New test tool
Testing a 64-bit hash algorithm for its collision rate has remained elusive for most. The sheer volume of data required to assess quality at this scale is too large for traditional test tools like
SMHasher. As a general guide, it requires 4 billion hashes to reach a 50% probability of getting a single collision. Accurate collision ratio evaluation requires many more hashes to actually measure something meaningful.A new open-source tool in
tests/collisionsoffers this capability. It requires a lot of memory to run, with a minimum of 32 GB to measure anything significant. But provided that one has a system with enough capacity, it can accurately measure the collision ratio of any 64-bit hash algorithm.Several algorithms were measured thanks to this tool, the result of which is currently consolidated on this wiki page. More can be added in the future.
This new development round also introduced several improvements to the
SMHashertest suite, uncovering new requirements for new scenarios. This proved beneficial to improve the general dispersion qualities ofxxh3andxxh128.Changelist
Here is a summarized list of changes for this version:
- perf: improved speed for large inputs (~+20%)
- perf: improved latency for small inputs (~10%)
- perf: s390x Vectorial code, by @easyaspi314
- cli: Improved support for Unicode filenames on Windows, thanks to @easyaspi314 and @t-mat
- api:
xxhash.hcan now be included in any order, multiple times, with and withoutXXH_STATIC_LINKING_ONLYorXXH_INLINE_ALL - build: xxHash's implementation has been transferred into
xxhash.h. There is no more need to havexxhash.cin the/includedirectory forXXH_INLINE_ALLto work - install: created pkg-config file, by @bket
- install: VCpkg installation instructions, by @LilyWangL
- doc: Highly improved code documentation, by @easyaspi314
- misc: New test tool in
/tests/collisions: brute force collision tester for 64-bit hashes
Downloads
-
Source code (ZIP)
0 downloads
-
Source code (TAR.GZ)
0 downloads
-
xxHash v0.7.2 Stable
released this
2019-10-08 08:11:56 -05:00 | 1471 commits to dev since this releaseThis a maintenance release, focused on the newer 128-bit variant.
Note thatXXH3is still labelled experimental : return values from this version are not comparable with other versions.- Fixed collision ratio of
XXH128for some specific input lengths, reported by @svpv - Improved
VSXandNEONvariants, by @easyaspi314 - Improved performance of scalar code path (
XXH_VECTOR=0), by @easyaspi314 xxhsum: can generate 128-bit hash with command-H2(note : for experimental purposes only !XXH128is not yet frozen)xxhsum: option-qremoves status notifications
Downloads
-
Source code (ZIP)
0 downloads
-
Source code (TAR.GZ)
0 downloads
- Fixed collision ratio of
-
xxHash v0.7.1 Stable
released this
2019-08-15 08:35:53 -05:00 | 1536 commits to dev since this releaseThe main feature of this release is an update of
XXH3, building upon many user feedbacks during this test period. The main points are :- Secret first : the algorithm computation can be altered by providing a "secret", which is any blob of bytes, of size >=
XXH3_SECRET_SIZE_MIN. seedis still available, and acts as a secret generator- As a consequence of these changes, note that new return values of
XXH3are not compatible with v0.7.0 - updated
ARM NEONvariant by @easyaspi314 - Streaming implementation is available
- Improve compatibility and performance with Visual Studio, with help from @aras-p
- Better integration when using
XXH_INLINE_ALL: do not pollute host namespace, use its own macros, such asXXH_ASSERT(),XXH_ALIGN, etc. - 128-bits variant provide helper function, for comparison of hashes.
Note that
XXH3is still considered experimental at this stage. It will have to remain stable for at least 2 releases before being branded "stable". After which stage, the algorithm and produced results will no longer evolve.Several general improvements are also present in this release :
- Better
clanggeneration ofrotlinstruction, thanks to @easyaspi314 XXH_REROLLbuild macro, to reduce binary size, by @easyaspi314- Improved
cmakescript, by @Mezozoysky - Full benchmark program provided in
/tests/bench
Downloads
-
Source code (ZIP)
0 downloads
-
Source code (TAR.GZ)
0 downloads
- Secret first : the algorithm computation can be altered by providing a "secret", which is any blob of bytes, of size >=
-
xxHash v0.7.0 Stable
released this
2019-03-15 12:15:30 -05:00 | 1776 commits to dev since this releaseThe main highlight of this release is the introduction of
XXH3, a new hash algorithm offering much improved speed, for both large and small inputs.XXH3is still labelled experimental, and must be unlocked with macroXXH_STATIC_LINKING_ONLY. The source code is located into its ownxxh3.hfile, which is automatically included (and therefore required) byxxhash.c. It's also possible to includexxh3.hdirectly, which will have a similar effect as triggeringXXH_INLINE_ALL.
At this stage,XXH3is suitable for ephemeral data and tests, but avoid storing long term hash values yet.
XXH3will be transferred into stable in a future release, after a period dedicated to gather users' feedback.For more details on
XXH3performance, see this article.note : there are known compilation issues under Visual Studio, which have been later fixed in
devbranch.Downloads
-
Source code (ZIP)
0 downloads
-
Source code (TAR.GZ)
0 downloads
-
Source code (ZIP)