ZXC is a fast lossless compression algorithm, targeting write-once, read-many workloads: data compressed once at build time, then decompressed on every device that reads it. It features an extremely fast decoder, with speeds of multiple GB/s per core: levels -1 to -6 decode 1.1x to 2.3x faster than LZ4 (lz4 --fast, lz4 or lz4hc, whichever matches the ratio) at an equal or better compression ratio. Level -7, the densest, compresses better than zstd -1 while decoding up to 2.9x faster.
ZXC also provides seekable archives for O(1) random access, in-place decompression, and dictionary compression for small data.
The ZXC format is fully specified in FORMAT.md and guarded by public conformance vectors. This repository is the reference implementation, provided as an open-source BSD 3-Clause licensed C library and a command line utility producing and decoding .zxc files, with official bindings for Rust, Python, Node.js, Go and WASM. The design is described in the whitepaper.
Decompression speedup against the closest competitor at each ratio tier β Silesia corpus (202 MB), single thread, reproducible with lzbench and TurboBench:
| Machine | -1 vs lz4 --fast |
-3 vs lz4 |
-6 vs lz4hc -9 |
-7 vs zstd -1 |
|---|---|---|---|---|
| Apple M3 | 2.35x | 2.23x | 1.68x | 2.93x |
| Axion (Neoverse-V2) | 1.82x | 1.51x | 1.26x | 2.16x |
| EPYC 9B45 (Zen 5) | 1.91x | 1.53x | 1.15x | 2.49x |
| EPYC 7B13 (Zen 3) | 1.65x | 1.28x | 1.09x | 2.36x |
ZXC also compresses smaller in each pairing: 61.76 % vs 61.90 %, 46.65 % vs 47.32 %, 36.28 % vs 36.75 % and 33.09 % vs 34.53 % of the original size.
Measured with lzbench 2.4.1 (by @inikep), MOREFLAGS="-march=native":
Apple M3 (Clang 21, macOS 27), Google Axion / Neoverse-V2 (GCC 14, GCP C4A), AMD EPYC 9B45 / Zen 5
(GCP C4D) and AMD EPYC 7B13 / Zen 3 (GCP C2D), x86 with SMT disabled. Re-run on every commit
(latest logs). TurboBench
carries ZXC alongside 70+ other codecs. Cycles per byte and memory:
whitepaper.
Apple M3 (ARM64)
| Codec | Compression | Decompression | Ratio |
|---|---|---|---|
| zxc 1.0.0 -1 | 982 MB/s | 14646 MB/s | 61.76 % |
| zxc 1.0.0 -2 | 652 MB/s | 12422 MB/s | 53.86 % |
| zxc 1.0.0 -3 | 290 MB/s | 11006 MB/s | 46.65 % |
| zxc 1.0.0 -4 | 211 MB/s | 9974 MB/s | 43.43 % |
| zxc 1.0.0 -5 | 127 MB/s | 9210 MB/s | 40.83 % |
| zxc 1.0.0 -6 | 16.6 MB/s | 7780 MB/s | 36.28 % |
| zxc 1.0.0 -7 | 11.2 MB/s | 5773 MB/s | 33.09 % |
| lz4 1.11.0 --fast -17 | 1468 MB/s | 6237 MB/s | 61.90 % |
| lz4 1.11.0 | 873 MB/s | 4937 MB/s | 47.32 % |
| lz4hc 1.11.0 -9 | 51.4 MB/s | 4629 MB/s | 36.75 % |
| lzav 7.1 -1 | 773 MB/s | 4268 MB/s | 39.23 % |
| snappy 1.3.1 | 722 MB/s | 3427 MB/s | 47.85 % |
| zstd 1.5.7 --fast --1 | 808 MB/s | 2778 MB/s | 41.01 % |
| zstd 1.5.7 -1 | 714 MB/s | 1971 MB/s | 34.53 % |
| zstd 1.5.7 -3 | 441 MB/s | 1877 MB/s | 31.20 % |
| zlib 1.4 -1 | 165 MB/s | 449 MB/s | 36.45 % |
Google Axion (Neoverse-V2, ARM64)
| Codec | Compression | Decompression | Ratio |
|---|---|---|---|
| zxc 1.0.0 -1 | 883 MB/s | 9394 MB/s | 61.76 % |
| zxc 1.0.0 -2 | 592 MB/s | 7764 MB/s | 53.86 % |
| zxc 1.0.0 -3 | 256 MB/s | 6489 MB/s | 46.65 % |
| zxc 1.0.0 -4 | 185 MB/s | 6043 MB/s | 43.43 % |
| zxc 1.0.0 -5 | 109 MB/s | 5614 MB/s | 40.83 % |
| zxc 1.0.0 -6 | 13.4 MB/s | 4891 MB/s | 36.28 % |
| zxc 1.0.0 -7 | 8.76 MB/s | 3539 MB/s | 33.09 % |
| lz4 1.11.0 --fast -17 | 1305 MB/s | 5158 MB/s | 61.90 % |
| lz4 1.11.0 | 748 MB/s | 4305 MB/s | 47.32 % |
| lz4hc 1.11.0 -9 | 42.6 MB/s | 3875 MB/s | 36.75 % |
| lzav 7.1 -1 | 642 MB/s | 2925 MB/s | 39.23 % |
| snappy 1.3.1 | 567 MB/s | 2292 MB/s | 47.85 % |
| zstd 1.5.7 --fast --1 | 603 MB/s | 2284 MB/s | 41.01 % |
| zstd 1.5.7 -1 | 523 MB/s | 1642 MB/s | 34.53 % |
| zstd 1.5.7 -3 | 325 MB/s | 1522 MB/s | 31.20 % |
| zlib 1.4 -1 | 115 MB/s | 389 MB/s | 36.45 % |
AMD EPYC 9B45 (Zen 5, x86_64)
| Codec | Compression | Decompression | Ratio |
|---|---|---|---|
| zxc 1.0.0 -1 | 873 MB/s | 11639 MB/s | 61.76 % |
| zxc 1.0.0 -2 | 588 MB/s | 10363 MB/s | 53.86 % |
| zxc 1.0.0 -3 | 253 MB/s | 8352 MB/s | 46.65 % |
| zxc 1.0.0 -4 | 184 MB/s | 7535 MB/s | 43.43 % |
| zxc 1.0.0 -5 | 110 MB/s | 7038 MB/s | 40.83 % |
| zxc 1.0.0 -6 | 15.3 MB/s | 6051 MB/s | 36.28 % |
| zxc 1.0.0 -7 | 10.2 MB/s | 4702 MB/s | 33.09 % |
| lz4 1.11.0 --fast -17 | 1321 MB/s | 6100 MB/s | 61.90 % |
| lz4 1.11.0 | 800 MB/s | 5445 MB/s | 47.32 % |
| lz4hc 1.11.0 -9 | 42.8 MB/s | 5274 MB/s | 36.75 % |
| lzav 7.1 -1 | 724 MB/s | 3687 MB/s | 39.23 % |
| snappy 1.3.1 | 564 MB/s | 2356 MB/s | 47.89 % |
| zstd 1.5.7 --fast --1 | 662 MB/s | 2429 MB/s | 41.01 % |
| zstd 1.5.7 -1 | 605 MB/s | 1885 MB/s | 34.53 % |
| zstd 1.5.7 -3 | 369 MB/s | 1724 MB/s | 31.20 % |
| zlib 1.4 -1 | 135 MB/s | 394 MB/s | 36.45 % |
AMD EPYC 7B13 (Zen 3, x86_64)
| Codec | Compression | Decompression | Ratio |
|---|---|---|---|
| zxc 1.0.0 -1 | 733 MB/s | 8175 MB/s | 61.76 % |
| zxc 1.0.0 -2 | 485 MB/s | 6796 MB/s | 53.86 % |
| zxc 1.0.0 -3 | 215 MB/s | 5255 MB/s | 46.65 % |
| zxc 1.0.0 -4 | 159 MB/s | 4959 MB/s | 43.43 % |
| zxc 1.0.0 -5 | 93.6 MB/s | 4788 MB/s | 40.83 % |
| zxc 1.0.0 -6 | 11.9 MB/s | 4303 MB/s | 36.28 % |
| zxc 1.0.0 -7 | 8.02 MB/s | 3153 MB/s | 33.09 % |
| lz4 1.11.0 --fast -17 | 1153 MB/s | 4968 MB/s | 61.90 % |
| lz4 1.11.0 | 675 MB/s | 4115 MB/s | 47.32 % |
| lz4hc 1.11.0 -9 | 35.2 MB/s | 3944 MB/s | 36.75 % |
| lzav 7.1 -1 | 509 MB/s | 3027 MB/s | 39.23 % |
| snappy 1.3.1 | 508 MB/s | 1838 MB/s | 47.89 % |
| zstd 1.5.7 --fast --1 | 486 MB/s | 1779 MB/s | 41.01 % |
| zstd 1.5.7 -1 | 442 MB/s | 1337 MB/s | 34.53 % |
| zstd 1.5.7 -3 | 233 MB/s | 1207 MB/s | 31.20 % |
| zlib 1.4 -1 | 106 MB/s | 355 MB/s | 36.45 % |
Reading the chart
Data compressed once is read many times over a link: disk, network, another core. With transfer and decoding overlapped, a reader gets
min(decode speed, bandwidth Γ· ratio)of original data per second. The chart divides that by the bandwidth, so 1x is sending the data uncompressed, and draws each codec at its best level for the link. Compression time is left out, as it is paid once.
- Below 0.4β0.6 GB/s the ratio decides, and
zstd -3is ahead.- From there on, a ZXC level delivers the most data: -7 up to 1.0β1.9 GB/s, then the faster levels in turn. No LZ4 or LZ4HC level is ever above the ZXC curve.
- Compression stops paying once one decoding thread is slower than the link: past 8.2β14.6 GB/s for
zxc -1, 5.0β6.2 GB/s for LZ4.
- Write once, read many. The encoder does the heavy lifting, so every device that reads the data decodes faster: content delivery, game assets, app bundles, firmware. Gains are largest on modern CPUs (benchmarks).
- O(1) random access. An optional seek table (
-S) decompresses any block without reading the rest. - Decodes in place. One buffer instead of two, zero allocations with a static context: made for firmware, FOTA and bootloaders (details).
- Small payloads too. A trained dictionary recovers ratio on 4β128 KB blocks (details).
- Runs everywhere. x86_64, ARM64, ARMv7, ARMv6, RISC-V, POWER, s390x, i386, with hand-tuned SIMD (AVX2/AVX-512/NEON).
- Production-grade. Continuously fuzzed by OSS-Fuzz, ASan/UBSan/Valgrind-clean, a specified wire format with conformance vectors, signed releases, BSD-3-Clause.
Used in ClickHouse (experimental column codec) Β· Packaged in Debian 14, Ubuntu 26.10, Homebrew, vcpkg, Conan, Winget Β· Benchmarked in lzbench & TurboBench
- Levels 1β2 (Fast): real-time assets: games, UI.
- Levels 3β4 (Balanced, 3 is the default): a better ratio than LZ4 at efficient compression speed.
- Level 5 (Compact): embedded and firmware: a better ratio than LZ4, much faster decode than Zstandard.
- Level 6 (Density): a better ratio and faster decode than LZ4HC on every measured platform; for archival, where compression time is amortized over many reads.
- Level 7 (Ultra): fills the gap between LZ4HC and Zstandard. It is denser than
zstd -1(33.09 % against 34.53 %) and decodes 2.2x to almost 3x faster. Compression is the slowest tier.
# Compress. -z is implied, and the output name defaults to <input>.zxc
zxc assets.tar # level 3 (default) -> assets.tar.zxc
zxc -z -5 assets.tar assets.tar.zxc # level 5
zxc -z -S assets.tar assets.tar.zxc # seekable: O(1) random-access decompression
# Decompress. "unzxc" (POSIX installs) is an alias for "zxc -d"
zxc -d assets.tar.zxc assets.tar
unzxc assets.tar.zxc assets.tar
# Decompress to stdout. "zxccat" (POSIX installs) runs "zxc -dcf" on each file
zxccat logs-1.zxc logs-2.zxc | grep ERROR
# Read a compressed file page by page in less (POSIX installs)
zxcless app.log.zxc
# Search compressed files with grep's options (POSIX installs)
zxcgrep -n ERROR logs-*.zxc
# Benchmark mode (testing speed on your machine)
zxc -b assets.tarEvery option is in zxc --help and the man page.
ZXC works as a drop-in external compressor for tar (reads stdin, writes stdout, returns 0 on success):
# GNU tar (Linux)
tar -I 'zxc -5' -cf archive.tar.zxc data/
tar -I 'zxc -d' -xf archive.tar.zxc
# bsdtar (macOS)
tar --use-compress-program='zxc -5' -cf archive.tar.zxc data/
tar --use-compress-program='zxc -d' -xf archive.tar.zxc
# Pipes (universal)
tar cf - data/ | zxc > archive.tar.zxc
zxc -d < archive.tar.zxc | tar xf -The one-shot calls are stateless and thread-safe: buffers are caller-allocated with explicit
bounds, and the simple forms take a level and nothing else. Every other setting is a parameter
on a reusable context, set by identifier, so call sites stay readable and forward-compatible.
Contexts are per-thread: never share a zxc_cctx / zxc_dctx between threads.
#include "zxc.h"
// One-shot: level 3, 512 KB blocks, no checksum, no dictionary
size_t bound = zxc_compress_bound(src_size);
int64_t compressed_size = zxc_compress(src, src_size, dst, bound, ZXC_LEVEL_DEFAULT);
int64_t decompressed_size = zxc_decompress(dst, (size_t)compressed_size, out, out_capacity);
// Anything else: configure a context (settings persist across calls)
zxc_cctx* cctx = zxc_create_cctx();
zxc_cctx_set_param(cctx, ZXC_C_LEVEL, ZXC_LEVEL_COMPACT);
zxc_cctx_set_param(cctx, ZXC_C_CHECKSUM, 1);
compressed_size = zxc_compress_cctx(cctx, src, src_size, dst, bound);
zxc_free_cctx(cctx);
zxc_dctx* dctx = zxc_create_dctx();
zxc_dctx_set_param(dctx, ZXC_D_CHECKSUM, 1); // verify what the archive carries
decompressed_size = zxc_decompress_dctx(dctx, dst, (size_t)compressed_size, out, out_capacity);
zxc_free_dctx(dctx);The same contexts configure zxc_stream_compress() / zxc_stream_decompress() (multi-threaded
file streaming, one worker per core by default; the context is read, its buffers unused), skip
per-call allocation in tight loops, carry a dictionary (zxc_cctx_set_dict()), and with
ZXC_C_SEEKABLE set to 1 append a seek table for O(1) random access. A setter refuses an
out-of-range value rather than clamp it, and 0 restores a parameter's default.
π See complete examples and advanced usage β stream API, reusable contexts, seekable readers, dictionaries and numeric pre-filters, as full compilable programs.
ZXC is packaged in the major ecosystems:
| Ecosystem | Install |
|---|---|
| vcpkg | vcpkg install zxc, or "dependencies": ["zxc"] in vcpkg.json |
| Conan | conan install -r conancenter --requires="zxc/[*]" --build=missing, or zxc/[*] under [requires] in conanfile.txt |
| Homebrew | brew install zxc |
| winget (Windows 10 1709+) | winget install hellobertrand.zxc |
The vcpkg, Conan Center and winget recipes are community-maintained and can lag a release behind; if one does, open an issue on that registry's index repository. Rust, Python, Node.js, Go and WASM packages: Language Bindings.
Prebuilt binaries for Linux, Windows and macOS (x86_64 and ARM64) are on the Releases page, with a signed checksum manifest and build attestations: verification steps.
find_package(zxc REQUIRED) # find_package(zxc CONFIG REQUIRED) via vcpkg or Conan
target_link_libraries(myapp PRIVATE zxc::zxc_lib)cc myapp.c $(pkg-config --cflags --libs libzxc) -o myappVendoring zxc instead β CMake FetchContent or add_subdirectory(), a Meson subproject or WrapDB β
and building from source, with the full option table and the PGO workflow:
docs/INSTALL.md.
Official bindings maintained in this repository:
| Language | Package Manager | Install Command | Documentation | Author |
|---|---|---|---|---|
| Rust | crates.io |
cargo add zxc-compress |
README | @hellobertrand |
| Python | PyPI |
pip install zxc-compress |
README | @nuberchardzer1 |
| Node.js | npm |
npm install zxc-compress |
README | @hellobertrand |
| Go | go get |
go get github.com/hellobertrand/zxc/bindings/go |
README | @hellobertrand |
| WASM | npm |
npm install zxc-wasm |
README | @hellobertrand |
Community-maintained bindings:
| Language | Package Manager | Install Command | Repository | Author |
|---|---|---|---|---|
| Go (native, no cgo) | pkg.go.dev | go get github.com/chmike/zxc-go |
https://github.com/chmike/zxc-go | @chmike |
| Nim | nimble | nimble install zxc |
https://github.com/openpeeps/zxc-nim | @georgelemon |
| Free Pascal | Build from source | Clone the repository | https://github.com/Xelitan/Free-Pascal-port-of-ZXC-compressor-decompressor | @Xelitan |
A normal decode needs two buffers, so peak memory is compressed + decompressed. ZXC can decode inside a single buffer: place the archive at its end, and the decoder writes the output from its start.
buffer: [ output β β β | archive β β β ]
^ write cursor ^ read cursor
Both cursors move right and a ZXC block never expands, so with a small margin the write cursor provably never catches the read cursor: peak memory drops to roughly decompressed alone. This is the shape of firmware images, FOTA payloads, bootloaders, game assets: anything read-only that already sits in RAM.
size_t need = zxc_decompress_inplace_bound(archive, archive_size); // reads header + footer only
uint8_t* buf = malloc(need);
memcpy(buf + (need - archive_size), archive, archive_size); // archive at the end
int64_t n = zxc_decompress_inplace(buf, need, archive_size); // decode from buf[0]
// buf[0 .. n) now holds the decompressed dataThe margin is about one block, so small payloads want small blocks (-B). An undersized buffer, or an archive whose output would reach input not yet read, is rejected with an error, never silent corruption; both entry points are fuzzed. For kernel code and heapless targets, zxc_decompress_inplace_dctx with a static context touches no allocator at all. Margin and context sizes per block size, and the static-context API: API.md.
For data compressed in small blocks (4 KBβ128 KB), a pre-trained dictionary substantially
improves the ratio. It prefills the LZ77 window at the start of each block, so the smaller the
block, the more it gains: a single small payload, or a large corpus split into small blocks for
random access. A .zxd also carries a shared literal Huffman table (levels 6β7), which spares small
blocks their own 128-byte table header.
Typical use cases: JSON API responses, small game assets, structured logs, key-value store records, RPC messages, seekable archives with small blocks.
# Train from a corpus of similar files. Without -o: ./dictionary_<dict_id>.zxd
zxc --train samples/*.json
zxc --train -o corpus.zxd samples/*.json # -o also accepts a directory
# The same dictionary is required to decompress: pass it with -D, there is no auto-lookup
zxc -z -D corpus.zxd input.json
zxc -d -D corpus.zxd input.json.zxcThe archive records only the dictionary's dict_id: decompressing without it returns
ZXC_ERROR_DICT_REQUIRED, with the wrong one ZXC_ERROR_DICT_MISMATCH. From C:
EXAMPLES.md and
API.md Β§11b. Wire format:
FORMAT.md Β§12.
Blocks are powers of two from 4 KB to 2 MB. The default, 512 KB, favours ratio and decode speed: each block starts with a cold match history, so larger blocks mean fewer cold starts. For streaming, embedded or memory-constrained use, 256 KB blocks halve the decompression context and cut the compression context by ~40 %, at a small cost in ratio and decode speed.
| Block Size | cctx memory | dctx memory | Ratio (level -3) | Decompression gain vs 256 KB |
|---|---|---|---|---|
| 256 KB | ~1.04 MB | ~514 KB | 47.21% | β |
| 512 KB (default) | ~1.79 MB | ~1.0 MB | 46.65% (β0.56 pp) | +1% to +8% depending on CPU |
# CLI β fall back to 256 KB blocks (e.g. embedded / streaming)
zxc -B 256K -5 input_file output_file
# API
zxc_cctx* cctx = zxc_create_cctx();
zxc_cctx_set_param(cctx, ZXC_C_LEVEL, ZXC_LEVEL_COMPACT);
zxc_cctx_set_param(cctx, ZXC_C_BLOCK_SIZE, 256 * 1024); // power of two, 4 KB..2 MBThe wire format is fully specified in docs/FORMAT.md (format version 9), so any third party can build an independent, interoperable decoder. Format v9 is frozen as of 1.0: every later release reads v9 archives. The pre-1.0 formats were clean breaks: v9 tools reject v8 archives, and docs/MIGRATION.md lists what changed and how to convert.
Two byte-frozen suites guard the format:
- Decoder conformance β
tests/conformance/ships public vectors per format version:valid/*.zxcstreams with their expected output, andinvalid/*.zxcstreams a correct decoder must reject, at least one per row of the format's error table. Point your own decoder at them; locally, run theconformanceCTest. - Encoder stability β
tests/format/pins the exact bytes the encoder emits for every block type and integrity field;vector-stability.ymlfails CI on any single-byte drift.
- Continuous fuzzing: Google OSS-Fuzz runs eight harnesses (roundtrip, decompress, block, in-place, push streaming, multi-threaded FILE* streaming, seekable, dictionary) around the clock, covering over 80 % of the library's lines (coverage history). The same harnesses run under ClusterFuzzLite (ASan + UBSan) on every pull request touching the library.
- Dynamic analysis: the CTest suite runs under ASan, UBSan and Valgrind in CI, with coverage tracked on Codecov.
- Static analysis: Cppcheck, Clang Static Analyzer, CodeQL and Snyk.
- Safe API: every operation takes an explicit buffer capacity.
ZXC Copyright Β© Bertrand Lebonnois and contributors. Licensed under the BSD 3-Clause License. See LICENSE for details; bundled third-party components are listed in THIRD_PARTY_NOTICES.
Third-Party Components:
- rapidhash by Nicolas De Carli (MIT) - Used for high-speed, platform-independent checksums.
Acknowledgements:
- PivCo-Huffman (PIVoted COding) by Marcin Ε»ukowski - the level-ordered layout and merge-based decode of ZXC's Huffman sections follow its design; implemented independently here. Special thanks to Dougall Johnson, whose idea of jointly nudging flat and length codes is behind the level 6 / 7 decode speedups.