• Stars
    star
    1,096
  • Rank 40,682 (Top 0.9 %)
  • Language
    C
  • License
    BSD 3-Clause "New...
  • Created about 15 years ago
  • Updated over 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Small strings compression library
SMAZ - compression for very small strings
-----------------------------------------

Smaz is a simple compression library suitable for compressing very short
strings. General purpose compression libraries will build the state needed
for compressing data dynamically, in order to be able to compress every kind
of data. This is a very good idea, but not for a specific problem: compressing
small strings will not work.

Smaz instead is not good for compressing general purpose data, but can compress
text by 40-50% in the average case (works better with English), and is able to
perform a bit of compression for HTML and urls as well. The important point is
that Smaz is able to compress even strings of two or three bytes!

For example the string "the" is compressed into a single byte.

To compare this with other libraries, think that like zlib will usually not be able to compress text shorter than 100 bytes.

COMPRESSION EXAMPLES
--------------------

'This is a small string' compressed by 50%
'foobar' compressed by 34%
'the end' compressed by 58%
'not-a-g00d-Exampl333' enlarged by 15%
'Smaz is a simple compression library' compressed by 39%
'Nothing is more difficult, and therefore more precious, than to be able to decide' compressed by 49%
'this is an example of what works very well with smaz' compressed by 49%
'1000 numbers 2000 will 10 20 30 compress very little' compressed by 10%

In general, lowercase English will work very well. It will suck with a lot
of numbers inside the strings. Other languages are compressed pretty well too,
the following is Italian, not very similar to English but still compressible
by smaz:

'Nel mezzo del cammin di nostra vita, mi ritrovai in una selva oscura' compressed by 33%
'Mi illumino di immenso' compressed by 37%
'L'autore di questa libreria vive in Sicilia' compressed by 28%

It can compress URLS pretty well:

'http://google.com' compressed by 59%
'http://programming.reddit.com' compressed by 52%
'http://github.com/antirez/smaz/tree/master' compressed by 46%

USAGE
-----

The lib consists of just two functions:

    int smaz_compress(char *in, int inlen, char *out, int outlen);

Compress the buffer 'in' of length 'inlen' and put the compressed data into
'out' of max length 'outlen' bytes. If the output buffer is too short to hold
the whole compressed string, outlen+1 is returned. Otherwise the length of the
compressed string (less then or equal to outlen) is returned.

    int smaz_decompress(char *in, int inlen, char *out, int outlen);

Decompress the buffer 'in' of length 'inlen' and put the decompressed data into
'out' of max length 'outlen' bytes. If the output buffer is too short to hold
the whole decompressed string, outlen+1 is returned. Otherwise the length of the
compressed string (less then or equal to outlen) is returned. This function will
not automatically put a nul-term at the end of the string if the original
compressed string didn't included a nulterm.


CREDITS
-------

Small was writte by Salvatore Sanfilippo and is released under the BSD license. Check the COPYING file for more information.

More Repositories

1

disque

Disque is a distributed message broker
C
7,969
star
2

kilo

A text editor in less than 1000 LOC with syntax highlight and search.
C
6,839
star
3

smallchat

A minimal programming example for a chat server
C
6,606
star
4

sds

Simple Dynamic Strings library for C
C
4,649
star
5

linenoise

A small self-contained alternative to readline and libedit
C
3,348
star
6

dump1090

Dump1090 is a simple Mode S decoder for RTLSDR devices
C
2,232
star
7

neural-redis

Neural networks module for Redis
C
2,218
star
8

lamernews

Lamer News -- an HN style social news site written in Ruby/Sinatra/Redis/JQuery
Ruby
1,357
star
9

hping

hping network tool
C
1,327
star
10

rax

A radix tree implementation in ANSI C
C
1,043
star
11

botlib

C Telegram bot framework
C
754
star
12

load81

SDL based Lua programming environment for kids similar to Codea
C
586
star
13

disque-module

Disque ported as Redis module
C
480
star
14

protoview

Flipper Zero app to display known and unknown signals
C
445
star
15

shapeme

Evolve images using simulated annealing
C
382
star
16

aocla

A small stack based, written to bring Advent of Code 2022 Day 13 puzzle to the extreme consequences
C
371
star
17

retwis

A Twitter-toy clone written in PHP and Redis, used in the early days to introduce Redis data types.
PHP
357
star
18

lua-cmsgpack

A self contained Lua MessagePack C implementation.
C
343
star
19

freakwan

A MicroPython driver for the SX1276 LoRa chip
C++
274
star
20

otree

a simple btree implementation with automatic space reclaiming
C
268
star
21

redis-sampler

Small program to understand the composition of your Redis data set
Ruby
260
star
22

redis-rb-cluster

Redis Cluster Ruby client based on redis-rb
Ruby
246
star
23

stonky

Stock market Telegram bot
C
242
star
24

RESP3

RESP protocol V3 repository. Contains the specification, and other related resource
221
star
25

redlock-rb

Redlock Redis-based distributed locks implementation in Ruby
Ruby
197
star
26

lloogg

LLOOGG realtime web log analyzer
PHP
195
star
27

redis-timeseries

Ruby library for Redis backed time series.
Ruby
195
star
28

redis-tools

Abandoned project "Redis tools". What was relevant is now part of redis-cli & redis-benchmark
C
194
star
29

pngtostl

Turn PNG images into STL 3D models that will "develop" in front of a light source
C
157
star
30

iconping

Icon Ping - visual ping to 4.2.2.2
Objective-C
129
star
31

jsrt

Javascript ray tracing engine
118
star
32

adventofcode2022

A few Advent of Code puzzles (2022 edition) in C
C
102
star
33

redimension

Redis multi-dimensional query library
Ruby
98
star
34

gopher2redis

A Ruby script that translates a directory structure and its files into the Redis keys to be served via Redis Gopher protocol
Ruby
90
star
35

mc-benchmark

Memcache port of Redis benchmark
C
85
star
36

listpack

A serialization format and implementation for backward-traversable lists of strings.
C
79
star
37

Bigdis

Bigdis - a file-based KV store speaking the Redis protocol
Tcl
76
star
38

visitors

Visitors fast web log analyzer
C
76
star
39

book-examples

Redis The Definitive Guide book code examples
Ruby
51
star
40

flipper-asteroids

Asteroids for Flipper Zero
C
50
star
41

Gitan

Gitan is a very basic web interface to create and inspect bare git repositories
Ruby
45
star
42

aspark

ASCII sparklines for the Enterprise
C
41
star
43

Jim

Jim is a small footprint Tcl interpreter, with some changes to the original language but mostly compatible.
40
star
44

nolate

NO LAme TEmplate System for Ruby
Ruby
40
star
45

recidiv

minimal continuous integration framework written in Tcl (used for Redis CI)
Tcl
38
star
46

tclircd

An IRC server I wrote in 2004 for fun, using the Tcl language.
Tcl
37
star
47

partitions

Partitions.tcl is a Tcl program to simulate partitions between physical hosts
Tcl
36
star
48

connect4-montecarlo

Simple connect 4 AI using Monte Carlo method
C
31
star
49

talk32

C program to talk via serial to MicroPython powered ESP32 boards
C
28
star
50

failed-3d-prints-bot

A Telegram bot that detects failed prints and send you an image of your printer
C
22
star
51

iqmeteo

Meteo widget for the Garmin Vivoactive HR powered by Yahoo Weather API
Shell
21
star
52

siphash

A modification of SipHash reference implementation to make it more practical for Redis usage
C
20
star
53

redisdotphp

Legacy Redis PHP client lib. A best-effort support repository.
18
star
54

yaku-ns

a DNS server I wrote 10 years ago. Here for historical reasons
C
18
star
55

strabo

Turns HGT elevation maps into 2D images or 3D models
C
18
star
56

nn-2003

2003 Neural Networks experiments -- when it was not mainstream ;-)
C
17
star
57

zx2040

RP2040 ZX Spectrum emulator
C
15
star
58

crack-checksum

Find checksum (crc8, xor, add) parameters in a set of messages.
C
13
star
59

hiredis

WARNING: hiredis repository moved to http://github.com/redis/hiredis. Just my private fork.
13
star
60

dict-scan-fuzz-tester

Fuzz testing for the SCAN underlying algorithm
C
11
star
61

Siboom

A simple markup system for writing books drafts
Ruby
10
star
62

sbignum

Old code about C library for big numbers plus Tcl bindings
C
9
star
63

codakido

Redirects to Load81 project
8
star
64

redis-cp-rewrite-sim

Redis/Raft snapshotting rewriting simulation
C
8
star
65

cache-mem-tester

Test memory efficiency of Redis / Memcached against values with a given size distribution.
Ruby
7
star
66

rascan

A prototype for a multi processes port scanner I wrote in 1998. Here only to archive it for myself.
C
6
star
67

LLM-FTC-sampling

First token cutoff sampling inference example
Python
6
star
68

t-watch-s3-micropython

Minimal MicroPython programming example for the Lilygo T-WATCH S3
Python
4
star
69

bma423-pure-mp

Pure MicroPython BMA423 accelerometer driver
Python
4
star
70

simple-language-model

Code for the video on feed-forward language model
Python
4
star
71

gif-pure-tcl

Pure TCL GIF generator
Tcl
4
star
72

bme680-pure-mp

Pure MicroPython Bosch BME680 sensor driver
Python
3
star
73

uc8151_micropython

UC8151 / IL0373 MicroPython e-paper display driver with support for greyscales and fast updates
Python
2
star
74

vl53l0x-nb

Fork of MicroPython driver for vl53l0x TOF sensor to add non-blocking mode.
Python
1
star
75

micropython-ft6x06

Simple driver for FT6x06 capacitive touch sensor in pure Python
Python
1
star