v1.5.4
版本发布时间: 2023-02-10 08:55:48
facebook/zstd最新发布版本:v1.5.6(2024-03-31 02:57:28)
Zstandard v1.5.4
is a pretty big release benefiting from one year of work, spread over > 650 commits. It offers significant performance improvements across multiple scenarios, as well as new features (detailed below). There is a crop of little bug fixes too, a few ones targeting the 32-bit mode are important enough to make this release a recommended upgrade.
Various Speed improvements
This release has accumulated a number of scenario-specific improvements, that cumulatively benefit a good portion of installed base in one way or another.
Among the easier ones to describe, the repository has received several contributions for arm
optimizations, notably from @JunHe77 and @danlark1. And @terrelln has improved decompression speed for non-x64 systems, including arm
. The combination of this work is visible in the following example, using an M1-Pro (aarch64
architecture) :
cpu | function | corpus | v1.5.2 |
v1.5.4 |
Improvement |
---|---|---|---|---|---|
M1 Pro | decompress | silesia.tar |
1370 MB/s | 1480 MB/s | + 8% |
Galaxy S22 | decompress | silesia.tar |
1150 MB/s | 1200 MB/s | + 4% |
Middle compression levels (5-12) receive some care too, with @terrelln improving the dispatch engine, and @danlark1 offering NEON
optimizations. Exact speed up vary depending on platform, cpu, compiler, and compression level, though one can expect gains ranging from +1 to +10% depending on scenarios.
cpu | function | corpus | v1.5.2 |
v1.5.4 |
Improvement |
---|---|---|---|---|---|
i7-9700k | compress -6 | silesia.tar |
110 MB/s | 121 MB/s | +10% |
Galaxy S22 | compress -6 | silesia.tar |
98 MB/s | 103 MB/s | +5% |
M1 Pro | compress -6 | silesia.tar |
122 MB/s | 130 MB/s | +6.5% |
i7-9700k | compress -9 | silesia.tar |
64 MB/s | 70 MB/s | +9.5% |
Galaxy S22 | compress -9 | silesia.tar |
51 MB/s | 52 MB/s | +1% |
M1 Pro | compress -9 | silesia.tar |
77 MB/s | 86 MB/s | +11.5% |
i7-9700k | compress -12 | silesia.tar |
31.6 MB/s | 31.8 MB/s | +0.5% |
Galaxy S22 | compress -12 | silesia.tar |
20.9 MB/s | 22.1 MB/s | +5% |
M1 Pro | compress -12 | silesia.tar |
36.1 MB/s | 39.7 MB/s | +10% |
Speed of the streaming compression interface has been improved by @embg in scenarios involving large files (where size is a multiple of the windowSize
parameter). The improvement is mostly perceptible at high speeds (i.e. ~level 1). In the following sample, the measurement is taken directly at ZSTD_compressStream()
function call, using a dedicated benchmark tool tests/fullbench
.
cpu | function | corpus | v1.5.2 |
v1.5.4 |
Improvement |
---|---|---|---|---|---|
i7-9700k | ZSTD_compressStream() -1 |
silesia.tar |
392 MB/s | 429 MB/s | +9.5% |
Galaxy S22 | ZSTD_compressStream() -1 |
silesia.tar |
380 MB/s | 430 MB/s | +13% |
M1 Pro | ZSTD_compressStream() -1 |
silesia.tar |
476 MB/s | 539 MB/s | +13% |
Finally, dictionary compression speed has received a good boost by @embg. Exact outcome varies depending on system and corpus. The following result is achieved by cutting the enwik8
compression corpus into 1KB blocks, generating a dictionary from these blocks, and then benchmarking the compression speed at level 1.
cpu | function | corpus | v1.5.2 |
v1.5.4 |
Improvement |
---|---|---|---|---|---|
i7-9700k | dictionary compress | enwik8 -B1K |
125 MB/s | 165 MB/s | +32% |
Galaxy S22 | dictionary compress | enwik8 -B1K |
138 MB/s | 166 MB/s | +20% |
M1 Pro | dictionary compress | enwik8 -B1K |
155 MB/s | 195 MB/s | +25 % |
There are a few more scenario-specifics improvements listed in the changelog
section below.
I/O Performance improvements
The 1.5.4 release improves IO performance of zstd
CLI, by using system buffers (macos
) and adding a new asynchronous I/O capability, enabled by default on large files (when threading is available). The user can also explicitly control this capability with the --[no-]asyncio
flag . These new threads remove the need to block on IO operations. The impact is mostly noticeable when decompressing large files (>= a few MBs), though exact outcome depends on environment and run conditions.
Decompression speed gets significant gains due to its single-threaded serial nature and the high speeds involved. In some cases we observe up to double performance improvement (local Mac machines) and a wide +15-45% benefit on Intel Linux servers (see table for details).
On the compression side of things, we’ve measured up to 5% improvements. The impact is lower because compression is already partially asynchronous via the internal MT mode (see release v1.3.4).
The following table shows the elapsed run time for decompressions of silesia
and enwik8
on several platforms - some Skylake-era Linux servers and an M1 MacbookPro. It compares the time it takes for version v1.5.2
to version v1.5.4
with asyncio on and off.
platform | corpus | v1.5.2 |
v1.5.4-no-asyncio |
v1.5.4 |
Improvement |
---|---|---|---|---|---|
Xeon D-2191A CentOS8 | enwik8 |
280 MB/s | 280 MB/s | 324 MB/s | +16% |
Xeon D-2191A CentOS8 | silesia.tar |
303 MB/s | 302 MB/s | 386 MB/s | +27% |
i7-1165g7 win10 | enwik8 |
270 MB/s | 280 MB/s | 350 MB/s | +27% |
i7-1165g7 win10 | silesia.tar |
450 MB/s | 440 MB/s | 580 MB/s | +28% |
i7-9700K Ubuntu20 | enwik8 |
600 MB/s | 604 MB/s | 829 MB/s | +38% |
i7-9700K Ubuntu20 | silesia.tar |
683 MB/s | 678 MB/s | 991 MB/s | +45% |
Galaxy S22 | enwik8 |
360 MB/s | 420 MB/s | 515 MB/s | +70% |
Galaxy S22 | silesia.tar |
310 MB/s | 320 MB/s | 580 MB/s | +85% |
MBP M1 | enwik8 |
428 MB/s | 734 MB/s | 815 MB/s | +90% |
MBP M1 | silesia.tar |
465 MB/s | 875 MB/s | 1001 MB/s | +115% |
Support of externally-defined sequence producers
libzstd
can now support external sequence producers via a new advanced registration function ZSTD_registerSequenceProducer()
(#3333).
This API allows users to provide their own custom sequence producer which libzstd invokes to process each block. The produced list of sequences (literals and matches) is then post-processed by libzstd to produce valid compressed blocks.
This block-level offload API is a more granular complement of the existing frame-level offload API compressSequences()
(introduced in v1.5.1
). It offers an easier migration story for applications already integrated with libzstd
: the user application continues to invoke the same compression functions ZSTD_compress2()
or ZSTD_compressStream2()
as usual, and transparently benefits from the specific properties of the external sequence producer. For example, the sequence producer could be tuned to take advantage of known characteristics of the input, to offer better speed / ratio.
One scenario that becomes possible is to combine this capability with hardware-accelerated matchfinders, such as the Intel® QuickAssist accelerator (Intel® QAT) provided in server CPUs such as the 4th Gen Intel® Xeon® Scalable processors (previously codenamed Sapphire Rapids). More details to be provided in future communications.
Change Log
perf: +20% faster huffman decompression for targets that can't compile x64 assembly (#3449, @terrelln)
perf: up to +10% faster streaming compression at levels 1-2 (#3114, @embg)
perf: +4-13% for levels 5-12 by optimizing function generation (#3295, @terrelln)
pref: +3-11% compression speed for arm
target (#3199, #3164, #3145, #3141, #3138, @JunHe77 and #3139, #3160, @danlark1)
perf: +5-30% faster dictionary compression at levels 1-4 (#3086, #3114, #3152, @embg)
perf: +10-20% cold dict compression speed by prefetching CDict tables (#3177, @embg)
perf: +1% faster compression by removing a branch in ZSTD_fast_noDict (#3129, @felixhandte)
perf: Small compression ratio improvements in high compression mode (#2983, #3391, @Cyan4973 and #3285, #3302, @daniellerozenblit)
perf: small speed improvement by better detecting STATIC_BMI2
for clang
(#3080, @TocarIP)
perf: Improved streaming performance when ZSTD_c_stableInBuffer
is set (#2974, @Cyan4973)
cli: Asynchronous I/O for improved cli speed (#2975, #2985, #3021, #3022, @yoniko)
cli: Change zstdless
behavior to align with zless
(#2909, @binhdvo)
cli: Keep original file if -c
or --stdout
is given (#3052, @dirkmueller)
cli: Keep original files when result is concatenated into a single output with -o
(#3450, @Cyan4973)
cli: Preserve Permissions and Ownership of regular files (#3432, @felixhandte)
cli: Print zlib/lz4/lzma library versions with -vv
(#3030, @terrelln)
cli: Print checksum value for single frame files with -lv
(#3332, @Cyan4973)
cli: Print dictID
when present with -lv
(#3184, @htnhan)
cli: when stderr
is not the console, disable status updates, but preserve final summary (#3458, @Cyan4973)
cli: support --best
and --no-name
in gzip
compatibility mode (#3059, @dirkmueller)
cli: support for posix
high resolution timer clock_gettime()
, for improved benchmark accuracy (#3423, @Cyan4973)
cli: improved help/usage (-h
, -H
) formatting (#3094, @dirkmueller and #3385, @jonpalmisc)
cli: Fix better handling of bogus numeric values (#3268, @ctkhanhly)
cli: Fix input consists of multiple files and stdin
(#3222, @yoniko)
cli: Fix tiny files passthrough (#3215, @cgbur)
cli: Fix for -r
on empty directory (#3027, @brailovich)
cli: Fix empty string as argument for --output-dir-*
(#3220, @embg)
cli: Fix decompression memory usage reported by -vv --long
(#3042, @u1f35c, and #3232, @zengyijing)
cli: Fix infinite loop when empty input is passed to trainer (#3081, @terrelln)
cli: Fix --adapt
doesn't work when --no-progress
is also set (#3354, @terrelln)
api: Support for External Sequence Producer (#3333, @embg)
api: Support for in-place decompression (#3432, @terrelln)
api: New ZSTD_CCtx_setCParams()
function, set all parameters defined in a ZSTD_compressionParameters
structure (#3403, @Cyan4973)
api: Streaming decompression detects incorrect header ID sooner (#3175, @Cyan4973)
api: Window size resizing optimization for edge case (#3345, @daniellerozenblit)
api: More accurate error codes for busy-loop scenarios (#3413, #3455, @Cyan4973)
api: Fix limit overflow in compressBound
and decompressBound
(#3362, #3373, Cyan4973) reported by @nigeltao
api: Deprecate several advanced experimental functions: streaming (#3408, @embg), copy (#3196, @mileshu)
bug: Fix corruption that rarely occurs in 32-bit mode with wlog=25 (#3361, @terrelln)
bug: Fix for block-splitter (#3033, @Cyan4973)
bug: Fixes for Sequence Compression API (#3023, #3040, @Cyan4973)
bug: Fix leaking thread handles on Windows (#3147, @animalize)
bug: Fix timing issues with cmake/meson builds (#3166, #3167, #3170, @Cyan4973)
build: Allow user to select legacy level for cmake (#3050, @shadchin)
build: Enable legacy support by default in cmake (#3079, @niamster)
build: Meson build script improvements (#3039, #3120, #3122, #3327, #3357, @eli-schwartz and #3276, @neheb)
build: Add aarch64 to supported architectures for zstd_trace (#3054, @ooosssososos)
build: support AIX architecture (#3219, @qiongsiwu)
build: Fix ZSTD_LIB_MINIFY
build macro, which now reduces static library size by half (#3366, @terrelln)
build: Fix Windows issues with Multithreading translation layer (#3364, #3380, @yoniko) and ARM64 target (#3320, @cwoffenden)
build: Fix cmake
script (#3382, #3392, @terrelln and #3252 @Tachi107 and #3167 @Cyan4973)
doc: Updated man page, providing more details for --train
mode (#3112, @Cyan4973)
doc: Add decompressor errata document (#3092, @terrelln)
misc: Enable Intel CET (#2992, #2994, @hjl-tools)
misc: Fix contrib/
seekable format (#3058, @yhoogstrate and #3346, @daniellerozenblit)
misc: Improve speed of the one-file library generator (#3241, @wahern and #3005, @cwoffenden)
PR list (generated by Github)
- x86-64: Enable Intel CET by @hjl-tools in https://github.com/facebook/zstd/pull/2992
- Add GitHub Action Checking that Zstd Runs Successfully Under CET by @felixhandte in https://github.com/facebook/zstd/pull/3015
- [opt] minor compression ratio improvement by @Cyan4973 in https://github.com/facebook/zstd/pull/2983
- Simplify HUF_decompress4X2_usingDTable_internal_bmi2_asm_loop by @WojciechMula in https://github.com/facebook/zstd/pull/3013
- Async write for decompression by @yoniko in https://github.com/facebook/zstd/pull/2975
- ZSTD CLI: Use buffered output by @yoniko in https://github.com/facebook/zstd/pull/2985
- Use faster Python script to amalgamate by @cwoffenden in https://github.com/facebook/zstd/pull/3005
- Change zstdless behavior to align with zless by @binhdvo in https://github.com/facebook/zstd/pull/2909
- AsyncIO compression part 1 - refactor of existing asyncio code by @yoniko in https://github.com/facebook/zstd/pull/3021
- Converge sumtype (offset | repcode) numeric representation towards offBase by @Cyan4973 in https://github.com/facebook/zstd/pull/2965
- fix sequence compression API in Explicit Delimiter mode by @Cyan4973 in https://github.com/facebook/zstd/pull/3023
- Lazy parameters adaptation (part 1 - ZSTD_c_stableInBuffer) by @Cyan4973 in https://github.com/facebook/zstd/pull/2974
- Print zlib/lz4/lzma library versions in verbose version output by @terrelln in https://github.com/facebook/zstd/pull/3030
- fix for -r on empty directory by @brailovich in https://github.com/facebook/zstd/pull/3027
- Add new CLI testing platform by @terrelln in https://github.com/facebook/zstd/pull/3020
- AsyncIO compression part 2 - added async read and asyncio to compression code by @yoniko in https://github.com/facebook/zstd/pull/3022
- Macos playtest envvars fix by @yoniko in https://github.com/facebook/zstd/pull/3035
- Fix required decompression memory usage reported by -vv + --long by @u1f35c in https://github.com/facebook/zstd/pull/3042
- Select legacy level for cmake by @shadchin in https://github.com/facebook/zstd/pull/3050
- [trace] Add aarch64 to supported architectures for zstd_trace by @ooosssososos in https://github.com/facebook/zstd/pull/3054
- New features for largeNbDicts benchmark by @embg in https://github.com/facebook/zstd/pull/3063
- Use helper function for bit manipulations. by @TocarIP in https://github.com/facebook/zstd/pull/3075
- [programs] Fix infinite loop when empty input is passed to trainer by @terrelln in https://github.com/facebook/zstd/pull/3081
- Enable STATIC_BMI2 for gcc/clang by @TocarIP in https://github.com/facebook/zstd/pull/3080
- build:cmake: enable ZSTD legacy support by default by @niamster in https://github.com/facebook/zstd/pull/3079
- Implement more gzip compatibility (#3037) by @dirkmueller in https://github.com/facebook/zstd/pull/3059
- [doc] Add decompressor errata document by @terrelln in https://github.com/facebook/zstd/pull/3092
- Handle newer less versions in zstdless testing by @dirkmueller in https://github.com/facebook/zstd/pull/3093
- [contrib][linux] Fix a warning in zstd_reset_cstream() by @cyberknight777 in https://github.com/facebook/zstd/pull/3088
- Software pipeline for ZSTD_compressBlock_fast_dictMatchState (+5-6% compression speed) by @embg in https://github.com/facebook/zstd/pull/3086
- Keep original file if -c or --stdout is given by @dirkmueller in https://github.com/facebook/zstd/pull/3052
- Split help in long and short version, cleanup formatting by @dirkmueller in https://github.com/facebook/zstd/pull/3094
- updated man page, providing more details for --train mode by @Cyan4973 in https://github.com/facebook/zstd/pull/3112
- Meson fixups for Windows by @eli-schwartz in https://github.com/facebook/zstd/pull/3039
- meson: for internal linkage, link to both libzstd and a static copy of it by @eli-schwartz in https://github.com/facebook/zstd/pull/3122
- Software pipeline for ZSTD_compressBlock_fast_extDict (+4-9% compression speed) by @embg in https://github.com/facebook/zstd/pull/3114
- ZSTD_fast_noDict: Avoid Safety Check When Writing
ip1
into Table by @felixhandte in https://github.com/facebook/zstd/pull/3129 - Correct and clarify repcode offset history logic by @embg in https://github.com/facebook/zstd/pull/3127
- [lazy] Optimize ZSTD_row_getMatchMask for levels 8-10 for ARM by @danlark1 in https://github.com/facebook/zstd/pull/3139
- fix leaking thread handles on Windows by @animalize in https://github.com/facebook/zstd/pull/3147
- Remove expensive assert in --rsyncable hot loop by @terrelln in https://github.com/facebook/zstd/pull/3154
- Bugfix for huge dictionaries by @embg in https://github.com/facebook/zstd/pull/3157
- common: apply two stage copy to aarch64 by @JunHe77 in https://github.com/facebook/zstd/pull/3145
- dec: adjust seqSymbol load on aarch64 by @JunHe77 in https://github.com/facebook/zstd/pull/3141
- Fix big endian ARM NEON path by @danlark1 in https://github.com/facebook/zstd/pull/3160
- [contrib] largeNbDicts bugfix + improvements by @embg in https://github.com/facebook/zstd/pull/3161
- display a warning message when using C90 clock_t by @Cyan4973 in https://github.com/facebook/zstd/pull/3166
- remove explicit standard setting from cmake script by @Cyan4973 in https://github.com/facebook/zstd/pull/3167
- removed gnu99 statement from meson recipe by @Cyan4973 in https://github.com/facebook/zstd/pull/3170
- "Short cache" optimization for level 1-4 DMS (+5-30% compression speed) by @embg in https://github.com/facebook/zstd/pull/3152
- Streaming decompression can detect incorrect header ID sooner by @Cyan4973 in https://github.com/facebook/zstd/pull/3175
- Add prefetchCDictTables CCtxParam (+10-20% cold dict compression speed) by @embg in https://github.com/facebook/zstd/pull/3177
- Fix ZSTD_BUILD_TESTS=ON with MSVC by @nocnokneo in https://github.com/facebook/zstd/pull/3180
- zstd -lv
to show dictID by @htnhan in https://github.com/facebook/zstd/pull/3184 - Intial commit to address 3090. Added support to decompress empty block. by @udayanbapat in https://github.com/facebook/zstd/pull/3118
- [largeNbDicts] Second try at fixing decompression segfault to always create compressInstructions by @zhuhan0 in https://github.com/facebook/zstd/pull/3209
- Clarify benchmark chunking docstring by @embg in https://github.com/facebook/zstd/pull/3197
- decomp: add prefetch for matched seq on aarch64 by @JunHe77 in https://github.com/facebook/zstd/pull/3164
- lib: add hint to generate more pipeline friendly code by @JunHe77 in https://github.com/facebook/zstd/pull/3138
- [AIX] Fix Compiler Flags and Bugs on AIX to Pass All Tests by @qiongsiwu in https://github.com/facebook/zstd/pull/3219
- zlibWrapper: Update for zlib 1.2.12 by @orbea in https://github.com/facebook/zstd/pull/3217
- Fix small file passthrough by @cgbur in https://github.com/facebook/zstd/pull/3215
- Add warning when multi-thread decompression is requested by @tomcwang in https://github.com/facebook/zstd/pull/3208
- stdin + multiple file fixes by @yoniko in https://github.com/facebook/zstd/pull/3222
- [AIX] Fixing hash4Ptr for Big Endian Systems by @qiongsiwu in https://github.com/facebook/zstd/pull/3227
- Disallow empty string as argument for --output-dir-flat and --output-dir-mirror by @embg in https://github.com/facebook/zstd/pull/3220
- Deprecate ZSTD_getDecompressedSize() by @terrelln in https://github.com/facebook/zstd/pull/3225
- [T124890272] Mark 2 Obsolete Functions(ZSTD_copy*Ctx) Deprecated in Zstd by @mileshu in https://github.com/facebook/zstd/pull/3196
- fileio_types.h : avoid dependency on mem.h by @Cyan4973 in https://github.com/facebook/zstd/pull/3232
- fixed: verbose output prints wrong value for
wlog
when doing--long
by @zengyijing in https://github.com/facebook/zstd/pull/3226 - Add explicit --pass-through flag and default to enabled for *cat by @terrelln in https://github.com/facebook/zstd/pull/3223
- Document pass-through behavior by @cgbur in https://github.com/facebook/zstd/pull/3242
- restore combine.sh bash performance while still sticking to POSIX by @wahern in https://github.com/facebook/zstd/pull/3241
- Benchmark program for sequence compression API by @embg in https://github.com/facebook/zstd/pull/3257
- streamline
make clean
list maintenance by adding aCLEAN
variable by @Cyan4973 in https://github.com/facebook/zstd/pull/3256 - drop
-E
flag insed
by @haampie in https://github.com/facebook/zstd/pull/3245 - compress:check more bytes to reduce
ZSTD_count
call by @JunHe77 in https://github.com/facebook/zstd/pull/3199 - build(cmake): improve pkg-config generation by @Tachi107 in https://github.com/facebook/zstd/pull/3252
- Fix for
zstd
CLI accepts bogus values for numeric parameters by @ctkhanhly in https://github.com/facebook/zstd/pull/3268 - ci: test pkg-config file by @Tachi107 in https://github.com/facebook/zstd/pull/3267
- Move ZSTD_DEPRECATED before ZSTDLIB_API/ZSTDLIB_STATIC_API for
clang
by @MaskRay in https://github.com/facebook/zstd/pull/3273 - Enable OpenSSF Scorecard Action by @felixhandte in https://github.com/facebook/zstd/pull/3277
- fixed zstd-pgo target for GCC by @ilyakurdyukov in https://github.com/facebook/zstd/pull/3281
- Cleaner threadPool initialization by @Cyan4973 in https://github.com/facebook/zstd/pull/3288
- Make fuzzing work without ZSTD_MULTITHREAD by @danlark1 in https://github.com/facebook/zstd/pull/3291
- Optimal huf depth by @daniellerozenblit in https://github.com/facebook/zstd/pull/3285
- Make ZSTD_getDictID_fromDDict() Read DictID from DDict by @felixhandte in https://github.com/facebook/zstd/pull/3290
- [contrib][linux-kernel] Generate SPDX license identifiers by @ojeda in https://github.com/facebook/zstd/pull/3294
- [lazy] Use switch instead of indirect function calls, improving compression speed by @terrelln in https://github.com/facebook/zstd/pull/3295
- [linux] Add zstd_common module by @terrelln in https://github.com/facebook/zstd/pull/3292
- Complete migration of ZSTD_c_enableLongDistanceMatching to ZSTD_paramSwitch_e framework by @embg in https://github.com/facebook/zstd/pull/3321
- meson: get version up front by @eli-schwartz in https://github.com/facebook/zstd/pull/3327
- Fix for MSVC C4267 warning on ARM64 (which becomes error C2220 with /WX) by @cwoffenden in https://github.com/facebook/zstd/pull/3320
- Enable dependabot for automatic GitHub Actions updates by @DimitriPapadopoulos in https://github.com/facebook/zstd/pull/3284
- Print checksum value for single frame files in cli with -v -l options by @Cyan4973 in https://github.com/facebook/zstd/pull/3332
- Fix window size resizing optimization for edge case by @daniellerozenblit in https://github.com/facebook/zstd/pull/3345
- [linux-kernel] Fix stack detection for newer gcc by @terrelln in https://github.com/facebook/zstd/pull/3348
- Reserve two fields in ZSTD_frameHeader by @embg in https://github.com/facebook/zstd/pull/3349
- Fix seekable format for empty string by @daniellerozenblit in https://github.com/facebook/zstd/pull/3346
- meson: make backtrace dependency on execinfo for musl libc compatibility by @neheb in https://github.com/facebook/zstd/pull/3276
- Refactor progress bar & summary line logic by @terrelln in https://github.com/facebook/zstd/pull/2984
- Use
__attribute__((aligned(1)))
for unaligned access by @Hello71 in https://github.com/facebook/zstd/pull/2881 - Separate parameter adaption from display update rate by @terrelln in https://github.com/facebook/zstd/pull/3354
- [decompress] Fix UB nullptr addition & improve fuzzer by @terrelln in https://github.com/facebook/zstd/pull/3356
- [legacy] Simplify legacy codebase by removing esoteric memory accesses and only use memcpy by @terrelln in https://github.com/facebook/zstd/pull/3355
- Fix corruption that rarely occurs in 32-bit mode with wlog=25 by @terrelln in https://github.com/facebook/zstd/pull/3361
- meson: partial fix for building pzstd on MSVC by @eli-schwartz in https://github.com/facebook/zstd/pull/3357
- [CI] Re-enable versions-test by @terrelln in https://github.com/facebook/zstd/pull/3371
- [api][visibility] Make the visibility macros more consistent by @terrelln in https://github.com/facebook/zstd/pull/3363
- [build] Fix ZSTD_LIB_MINIFY build option by @terrelln in https://github.com/facebook/zstd/pull/3366
- [zdict] Fix static linking only include guards by @terrelln in https://github.com/facebook/zstd/pull/3372
- check potential overflow of compressBound() by @Cyan4973 in https://github.com/facebook/zstd/pull/3362
- decompressBound tests and fix by @Cyan4973 in https://github.com/facebook/zstd/pull/3373
- Meson test fixups by @eli-schwartz in https://github.com/facebook/zstd/pull/3120
- [pzstd] Fixes for Windows build by @terrelln in https://github.com/facebook/zstd/pull/3380
- Windows MT layer bug fixes by @yoniko in https://github.com/facebook/zstd/pull/3364
- Update Copyright Comments by @felixhandte in https://github.com/facebook/zstd/pull/3173
- [docs] Clarify dictionary loading documentation by @terrelln in https://github.com/facebook/zstd/pull/3381
- [build][cmake] Fix cmake with custom assembler by @terrelln in https://github.com/facebook/zstd/pull/3382
- Pin actions/checkout Dependency to Specific Commit Hash by @felixhandte in https://github.com/facebook/zstd/pull/3384
- Improve help/usage (
-h
,-H
) formatting by @jonpalmisc in https://github.com/facebook/zstd/pull/3385 - [cmake] Add noexecstack to compiler/linker flags by @terrelln in https://github.com/facebook/zstd/pull/3392
- Fix
-Wdocumentation
by @terrelln in https://github.com/facebook/zstd/pull/3393 - Support decompression of compressed blocks of size ZSTD_BLOCKSIZE_MAX by @Cyan4973 in https://github.com/facebook/zstd/pull/3399
- spec update : require minimum nb of literals for 4-streams mode by @Cyan4973 in https://github.com/facebook/zstd/pull/3398
- External matchfinder API by @embg in https://github.com/facebook/zstd/pull/3333
- New
ZSTD_CCtx_setCParams()
entry point, to set all parameters defined in aZSTD_compressionParameters
structure by @Cyan4973 in https://github.com/facebook/zstd/pull/3403 - Move deprecated annotation before static to allow C++ compilation for clang by @danlark1 in https://github.com/facebook/zstd/pull/3400
- Optimal huff depth speed improvements by @daniellerozenblit in https://github.com/facebook/zstd/pull/3302
- improve compression ratio of small alphabets by @Cyan4973 in https://github.com/facebook/zstd/pull/3391
- Fix fuzzing with ZSTD_MULTITHREAD by @danlark1 in https://github.com/facebook/zstd/pull/3417
- minor refactoring for timefn by @Cyan4973 in https://github.com/facebook/zstd/pull/3413
- Add support for in-place decompression by @terrelln in https://github.com/facebook/zstd/pull/3421
- fix when nb of literals is very small by @Cyan4973 in https://github.com/facebook/zstd/pull/3419
- Deprecate advanced streaming functions by @embg in https://github.com/facebook/zstd/pull/3408
- Disable Custom ASAN/MSAN Poisoning on MinGW Builds by @felixhandte in https://github.com/facebook/zstd/pull/3424
- [tests] Fix version test determinism by @terrelln in https://github.com/facebook/zstd/pull/3422
- Refactor
timefn
unit, restore support forclock_gettime()
by @Cyan4973 in https://github.com/facebook/zstd/pull/3423 - Fuzz on maxBlockSize by @daniellerozenblit in https://github.com/facebook/zstd/pull/3418
- Fuzz the External Matchfinder API by @embg in https://github.com/facebook/zstd/pull/3437
- Cap hashLog & chainLog to ensure that we only use 32 bits of hash by @terrelln in https://github.com/facebook/zstd/pull/3438
- [versions-test] Work around bug in dictionary builder for older versions by @terrelln in https://github.com/facebook/zstd/pull/3436
- added c89 build test to CI by @Cyan4973 in https://github.com/facebook/zstd/pull/3435
- added cygwin tests to Github Actions by @Cyan4973 in https://github.com/facebook/zstd/pull/3431
- Huffman refactor by @terrelln in https://github.com/facebook/zstd/pull/3434
- Fix bufferless API with attached dictionary by @terrelln in https://github.com/facebook/zstd/pull/3441
- Test PGO Builds by @felixhandte in https://github.com/facebook/zstd/pull/3442
- Fix CLI Handling of Permissions and Ownership by @felixhandte in https://github.com/facebook/zstd/pull/3432
- Fix -Wstringop-overflow warning by @terrelln in https://github.com/facebook/zstd/pull/3440
- refactor : --rm ignored with stdout by @Cyan4973 in https://github.com/facebook/zstd/pull/3443
- Fix sequence validation and seqStore bounds check by @daniellerozenblit in https://github.com/facebook/zstd/pull/3439
- Fix ZSTD_estimate* and ZSTD_initCStream() docs by @embg in https://github.com/facebook/zstd/pull/3448
- Fix 32-bit build errors in zstd seekable format by @daniellerozenblit in https://github.com/facebook/zstd/pull/3452
- Fuzz large offsets through sequence compression api by @daniellerozenblit in https://github.com/facebook/zstd/pull/3447
- [huf] Add generic C versions of the fast decoding loops by @terrelln in https://github.com/facebook/zstd/pull/3449
- Provide more accurate error codes for busy-loop scenarios by @Cyan4973 in https://github.com/facebook/zstd/pull/3455
- disable --rm on -o command by @Cyan4973 in https://github.com/facebook/zstd/pull/3450
- [Bugfix] CLI row hash flags set the wrong values by @yoniko in https://github.com/facebook/zstd/pull/3457
- [huf] Fix bug in fast C decoders by @terrelln in https://github.com/facebook/zstd/pull/3459
- Disable status updates when
stderr
is not the console by @Cyan4973 in https://github.com/facebook/zstd/pull/3458 - fix long offset resolution by @daniellerozenblit in https://github.com/facebook/zstd/pull/3460
- Simplify 32-bit long offsets decoding logic by @terrelln in https://github.com/facebook/zstd/pull/3467
New Contributors
- @WojciechMula made their first contribution in https://github.com/facebook/zstd/pull/3013
- @trixirt made their first contribution in https://github.com/facebook/zstd/pull/3026
- @brailovich made their first contribution in https://github.com/facebook/zstd/pull/3027
- @u1f35c made their first contribution in https://github.com/facebook/zstd/pull/3042
- @shadchin made their first contribution in https://github.com/facebook/zstd/pull/3050
- @ooosssososos made their first contribution in https://github.com/facebook/zstd/pull/3054
- @TocarIP made their first contribution in https://github.com/facebook/zstd/pull/3075
- @xry111 made their first contribution in https://github.com/facebook/zstd/pull/3084
- @niamster made their first contribution in https://github.com/facebook/zstd/pull/3079
- @dirkmueller made their first contribution in https://github.com/facebook/zstd/pull/3059
- @cyberknight777 made their first contribution in https://github.com/facebook/zstd/pull/3088
- @dpelle made their first contribution in https://github.com/facebook/zstd/pull/3095
- @paulmenzel made their first contribution in https://github.com/facebook/zstd/pull/3108
- @cuishuang made their first contribution in https://github.com/facebook/zstd/pull/3117
- @averred made their first contribution in https://github.com/facebook/zstd/pull/3135
- @JunHe77 made their first contribution in https://github.com/facebook/zstd/pull/3145
- @htnhan made their first contribution in https://github.com/facebook/zstd/pull/3184
- @udayanbapat made their first contribution in https://github.com/facebook/zstd/pull/3118
- @zhuhan0 made their first contribution in https://github.com/facebook/zstd/pull/3205
- @mgord9518 made their first contribution in https://github.com/facebook/zstd/pull/3218
- @qiongsiwu made their first contribution in https://github.com/facebook/zstd/pull/3219
- @orbea made their first contribution in https://github.com/facebook/zstd/pull/3217
- @cgbur made their first contribution in https://github.com/facebook/zstd/pull/3215
- @tomcwang made their first contribution in https://github.com/facebook/zstd/pull/3208
- @mileshu made their first contribution in https://github.com/facebook/zstd/pull/3196
- @zengyijing made their first contribution in https://github.com/facebook/zstd/pull/3226
- @grossws made their first contribution in https://github.com/facebook/zstd/pull/3230
- @wahern made their first contribution in https://github.com/facebook/zstd/pull/3241
- @daniellerozenblit made their first contribution in https://github.com/facebook/zstd/pull/3258
- @DimitriPapadopoulos made their first contribution in https://github.com/facebook/zstd/pull/3259
- @sashashura made their first contribution in https://github.com/facebook/zstd/pull/3264
- @haampie made their first contribution in https://github.com/facebook/zstd/pull/3247
- @Tachi107 made their first contribution in https://github.com/facebook/zstd/pull/3252
- @ctkhanhly made their first contribution in https://github.com/facebook/zstd/pull/3268
- @MaskRay made their first contribution in https://github.com/facebook/zstd/pull/3273
- @ilyakurdyukov made their first contribution in https://github.com/facebook/zstd/pull/3281
- @ojeda made their first contribution in https://github.com/facebook/zstd/pull/3294
- @GermanAizek made their first contribution in https://github.com/facebook/zstd/pull/3304
- @joycebrum made their first contribution in https://github.com/facebook/zstd/pull/3309
- @yiyuaner made their first contribution in https://github.com/facebook/zstd/pull/3300
- @nmoinvaz made their first contribution in https://github.com/facebook/zstd/pull/3289
- @jonpalmisc made their first contribution in https://github.com/facebook/zstd/pull/3385
Full Automated Changelog: https://github.com/facebook/zstd/compare/v1.5.2...v1.5.4
1、 zstd-1.5.4.tar.gz 2.06MB
2、 zstd-1.5.4.tar.gz.sha256 84B
3、 zstd-1.5.4.tar.gz.sig 858B
4、 zstd-1.5.4.tar.zst 1.55MB
5、 zstd-1.5.4.tar.zst.sha256 85B
6、 zstd-1.5.4.tar.zst.sig 858B
7、 zstd-v1.5.4-win64.zip 1.65MB