| inikep | ab2f770 | 2016-08-25 10:07:20 +0200 | [diff] [blame] | 1 | zstd - Command Line Interface |
| 2 | ================================ |
| 3 | |
| 4 | Command Line Interface (CLI) can be created using the `make` command without any additional parameters. |
| 5 | There are however other Makefile targets that create different variations of CLI: |
| 6 | - `zstd` : default CLI supporting gzip-like arguments; includes dictionary builder, benchmark, and support for decompression of legacy zstd versions |
| 7 | - `zstd32` : Same as `zstd`, but forced to compile in 32-bits mode |
| 8 | - `zstd_nolegacy` : Same as `zstd` except of support for decompression of legacy zstd versions |
| 9 | - `zstd-small` : CLI optimized for minimal size; without dictionary builder, benchmark, and support for decompression of legacy zstd versions |
| 10 | - `zstd-compress` : compressor-only version of CLI; without dictionary builder, benchmark, and support for decompression of legacy zstd versions |
| 11 | - `zstd-decompress` : decompressor-only version of CLI; without dictionary builder, benchmark, and support for decompression of legacy zstd versions |
| 12 | |
| 13 | |
| 14 | #### Aggregation of parameters |
| 15 | CLI supports aggregation of parameters i.e. `-b1`, `-e18`, and `-i1` can be joined into `-b1e18i1`. |
| 16 | |
| 17 | |
| 18 | #### Dictionary builder in Command Line Interface |
| 19 | Zstd offers a training mode, which can be used to tune the algorithm for a selected |
| 20 | type of data, by providing it with a few samples. The result of the training is stored |
| 21 | in a file selected with the `-o` option (default name is `dictionary`), |
| 22 | which can be loaded before compression and decompression. |
| 23 | |
| 24 | Using a dictionary, the compression ratio achievable on small data improves dramatically. |
| 25 | These compression gains are achieved while simultaneously providing faster compression and decompression speeds. |
| 26 | Dictionary work if there is some correlation in a family of small data (there is no universal dictionary). |
| 27 | Hence, deploying one dictionary per type of data will provide the greater benefits. |
| 28 | Dictionary gains are mostly effective in the first few KB. Then, the compression algorithm |
| 29 | will rely more and more on previously decoded content to compress the rest of the file. |
| 30 | |
| 31 | Usage of the dictionary builder and created dictionaries with CLI: |
| inikep | 637d335 | 2016-08-25 10:42:49 +0200 | [diff] [blame^] | 32 | 1. Create the dictionary : `zstd --train FullPathToTrainingSet/* -o dictionaryName` |
| 33 | 2. Compress with dictionary: `zstd FILE -D dictionaryName` |
| 34 | 3. Decompress with dictionary: `zstd --decompress FILE.zst -D dictionaryName` |
| inikep | ab2f770 | 2016-08-25 10:07:20 +0200 | [diff] [blame] | 35 | |
| 36 | |
| 37 | |
| 38 | #### Benchmark in Command Line Interface |
| 39 | CLI includes in-memory compression benchmark module for zstd. |
| inikep | 637d335 | 2016-08-25 10:42:49 +0200 | [diff] [blame^] | 40 | The benchmark is conducted using given filenames. The files are read into memory and joined together. |
| inikep | ab2f770 | 2016-08-25 10:07:20 +0200 | [diff] [blame] | 41 | It makes benchmark more precise as it eliminates I/O overhead. |
| 42 | Many filenames can be supplied as multiple parameters, parameters with wildcards or |
| 43 | names of directories can be used as parameters with the `-r` option. |
| 44 | |
| 45 | The benchmark measures ratio, compressed size, compression and decompression speed. |
| 46 | One can select compression levels starting from `-b` and ending with `-e`. |
| 47 | The `-i` parameter selects minimal time used for each of tested levels. |
| 48 | |
| 49 | |
| 50 | |
| 51 | #### Usage of Command Line Interface |
| 52 | The full list of options can be obtained with `-h` or `-H` parameter: |
| 53 | ``` |
| 54 | Usage : |
| 55 | zstd [args] [FILE(s)] [-o file] |
| 56 | |
| 57 | FILE : a filename |
| 58 | with no FILE, or when FILE is - , read standard input |
| 59 | Arguments : |
| 60 | -# : # compression level (1-19, default:3) |
| 61 | -d : decompression |
| 62 | -D file: use `file` as Dictionary |
| 63 | -o file: result stored into `file` (only if 1 input file) |
| 64 | -f : overwrite output without prompting |
| 65 | --rm : remove source file(s) after successful de/compression |
| 66 | -k : preserve source file(s) (default) |
| 67 | -h/-H : display help/long help and exit |
| 68 | |
| 69 | Advanced arguments : |
| 70 | -V : display Version number and exit |
| 71 | -v : verbose mode; specify multiple times to increase log level (default:2) |
| 72 | -q : suppress warnings; specify twice to suppress errors too |
| 73 | -c : force write to standard output, even if it is the console |
| 74 | -r : operate recursively on directories |
| 75 | --ultra : enable levels beyond 19, up to 22 (requires more memory) |
| 76 | --no-dictID : don't write dictID into header (dictionary compression) |
| 77 | --[no-]check : integrity check (default:enabled) |
| 78 | --test : test compressed file integrity |
| 79 | --[no-]sparse : sparse mode (default:enabled on file, disabled on stdout) |
| 80 | |
| 81 | Dictionary builder : |
| 82 | --train ## : create a dictionary from a training set of files |
| 83 | -o file : `file` is dictionary name (default: dictionary) |
| 84 | --maxdict ## : limit dictionary to specified size (default : 112640) |
| 85 | -s# : dictionary selectivity level (default: 9) |
| 86 | --dictID ## : force dictionary ID to specified value (default: random) |
| 87 | |
| 88 | Benchmark arguments : |
| 89 | -b# : benchmark file(s), using # compression level (default : 1) |
| 90 | -e# : test all compression levels from -bX to # (default: 1) |
| 91 | -i# : minimum evaluation time in seconds (default : 3s) |
| 92 | -B# : cut file into independent blocks of size # (default: no block) |
| 93 | ``` |