Snappy (compression)

Snappy (previously known as Zippy) is a fast data compression and decompression library written in C++ by Google based on ideas from LZ77 and open-sourced in 2011. It does not aim for maximum compression, or compatibility with any other compression library; instead, it aims for very high speeds and reasonable compression. Compression speed is 250 MB/s and decompression speed is 500 MB/s using a single core of a Core i7 processor running in 64-bit mode. The compression ratio is 20–100% lower than gzip.

Snappy is widely used in Google projects like BigTable, MapReduce and in compression data in Google's internal RPC systems. It can be used in open-source projects like MariaDB ColumnStore,Cassandra, Hadoop, LevelDB, MongoDB, RocksDB, Lucene. Decompression is tested to detect any errors in the compressed stream. Snappy does not use inline assembler and is portable.

Snappy encoding is not bit-oriented, but byte-oriented (only whole bytes are emitted or consumed from a stream). The format uses no entropy encoder, like Huffman tree or arithmetic encoder.

The first bytes of the stream are the length of uncompressed data, stored as a little-endian varint, which allows for variable-length encoding. The lower seven bits of each byte are used for data and the high bit is a flag to indicate the end of the length field.