inikep/lzbench

Introduction

lzbench is an in-memory benchmark of open-source LZ77/LZSS/LZMA compressors. It joins all compressors into a single exe. At the beginning an input file is read to memory. Then all compressors are used to compress and decompress the file and decompressed file is verified. This approach has a big advantage of using the same compiler with the same optimizations for all compressors. The disadvantage is that it requires source code of each compressor (therefore Slug or lzturbo are not included).

Status

Usage

usage: lzbench [options] input [input2] [input3]

where [input] is a file or a directory and [options] are:
 -b#   set block/chunk size to # KB (default = MIN(filesize,1747626 KB))
 -c#   sort results by column # (1=algname, 2=ctime, 3=dtime, 4=comprsize)
 -e#   #=compressors separated by '/' with parameters specified after ',' (deflt=fast)
 -iX,Y set min. number of compression and decompression iterations (default = 1, 1)
 -j    join files in memory but compress them independently (for many small files)
 -l    list of available compressors and aliases
 -m#   set memory limit to # MB (default = no limit)
 -o#   output text format 1=Markdown, 2=text, 3=text+origSize, 4=CSV (default = 2)
 -p#   print time for all iterations: 1=fastest 2=average 3=median (default = 1)
 -r    operate recursively on directories
 -s#   use only compressors with compression speed over # MB (default = 0 MB)
 -tX,Y set min. time in seconds for compression and decompression (default = 1, 2)
 -v    disable progress information
 -x    disable real-time process priority
 -z    show (de)compression times instead of speed

Example usage:
  lzbench -ezstd filename = selects all levels of zstd
  lzbench -ebrotli,2,5/zstd filename = selects levels 2 & 5 of brotli and zstd
  lzbench -t3 -u5 fname = 3 sec compression and 5 sec decompression loops
  lzbench -t0 -u0 -i3 -j5 -ezstd fname = 3 compression and 5 decompression iter.
  lzbench -t0u0i3j5 -ezstd fname = the same as above with aggregated parameters

Compilation

For Linux/MacOS/MinGW (Windows):

make

For 32-bit compilation:

make BUILD_ARCH=32-bit

The default linking for Linux is dynamic and static for Windows. This can be changed with make BUILD_STATIC=0/1.

To remove one of compressors you can add -DBENCH_REMOVE_XXX to DEFINES in Makefile (e.g. DEFINES += -DBENCH_REMOVE_LZ4 to remove LZ4). You also have to remove corresponding *.o files (e.g. lz4/lz4.o and lz4/lz4hc.o).

lzbench was tested with:

• Ubuntu: gcc 4.8 (both 32-bit and 64-bit), 4.9, 5 (32-bit and 64-bit), 6 (32-bit and 64-bit), 7, 8, 9 and clang 3.5, 3.6, 3.8, 3.9, 4.0, 5.0, 6.0, 7, 8, 9
• MacOS: Apple LLVM version 9.1.0
• MinGW (Windows): gcc 5.3 (32-bit), gcc 6.2 (both 32-bit and 64-bit), gcc 9.1

Supported compressors

Warning: some of the compressors listed here have security issues and/or are no longer maintained. For information about the security of the various compressors, see the CompFuzz Results page.

• blosclz 2.0.0
• brieflz 1.3.0
• brotli 1.0.9
• bzip2 1.0.8
• crush 1.0
• csc 2016-10-13 - WARNING: it can throw SEGFAULT compiled with Apple LLVM version 7.3.0 (clang-703.0.31)
• density 0.14.2 - WARNING: it contains bugs (shortened decompressed output))
• fastlz 0.5.0
• fast-lzma2 1.0.1
• gipfeli 2016-07-13
• glza 0.8
• libdeflate v1.6
• lizard v1.0 (formerly lz5)
• lz4/lz4hc v1.9.3
• lzf 3.6
• lzfse/lzvn 1.0
• lzg 1.0.10
• lzham 1.0 lzjb 2010
• lzlib 1.12-rc2
• lzma v19.00
• lzmat 1.01 v1.0 - WARNING: it contains bugs (decompression error; returns 0); it can throw SEGFAULT compiled with gcc 4.9+ -O3
• lzo 2.10
• lzrw 15-Jul-1991
• lzsse 2019-04-18 (1847c3e827)
• pithy 2011-12-24 - WARNING: it contains bugs (decompression error; returns 0)
• quicklz 1.5.0
• shrinker 0.1 - WARNING: it can throw SEGFAULT compiled with gcc 4.9+ -O3
• slz 1.2.0 - only a compressor, uses zlib for decompression
• snappy 2020-07-11 (4dd277f)
• tornado 0.6a
• ucl 1.03
• wflz 2015-09-16 - WARNING: it can throw SEGFAULT compiled with gcc 4.9+ -O3
• xpack 2016-06-02
• xz 5.2.4
• yalz77 2015-09-19
• yappy 2014-03-22 - WARNING: fails to decompress properly on ARM
• zlib 1.2.11
• zling 2018-10-12 - according to the author using libzling in a production environment is not a good idea
• zstd 1.5.0
• nvcomp 1.2.3 - If CUDA is available.

CUDA support

If CUDA is available, lzbench supports additional compressors:

• cudaMemcpy - similar to the reference memcpy benchmark, using GPU memory
• nvcomp 1.2.2 LZ4 GPU-only compressor

The directory where the CUDA compiler and libraries are available can be passed to make via the CUDA_BASE variable, e.g.:

make CUDA_BASE=/usr/local/cuda

Benchmarks

The following results are obtained with lzbench 1.8 with the -t16,16 -eall options using 1 core of Intel Core i7-8700K, Ubuntu 18.04.3 64-bit, and clang 9.0.1 with "silesia.tar" which contains tarred files from Silesia compression corpus. The results sorted by ratio are available here.