Experiments

• Synopsis
• Lorenzo, the default
• Composing a command line
  • Compression
  • Decompression
• Spline
• The LC path
• The FZG path
• Checking a run is actually lossless-to-bound
• Skipping stages

Synopsis

# compression (-z)
cusz -t <type> -m <mode> -e <error bound> -i <file> -l <N-D size> -z [--report cr]
# decompression (-x)
cusz -i <.cusza file> -x [--compare <original file>]

<...> marks a required value; [...] marks optional. Type cusz or cusz -h for the full built-in help. Flag-by-flag tables are under Composing a command line below.

Set up once:

export PATH=$(pwd)/bin:$PATH               ## specify the path temporarily
cd data && sh ./sh.get-sample-data         ## download sample data
export CESM=$(pwd)/cesm-CLDHGH-3600x1800
export EB=1e-4

${CESM} is 2D, 3600-1800, f32. Substitute your own file and -l extent throughout; nothing below is specific to this dataset.

Lorenzo, the default

cusz -t f32 -m r2r -e ${EB} -i ${CESM} -l 3600-1800 -z --report cr
cusz -i ${CESM}.cusza -x --compare ${CESM}

-p lrz is the default and may be stated explicitly. Error control is -m r2r (relative to value range) or -m abs; -e is the bound.

Composing a command line

The free form of composing a command carrying different options is essentially reflected in compression.

Compression

• Subcommand form: cusz compress or cusz zip
• Command option form: cusz -z or cusz --zip

Part	Option	Long form	Values
input	-i	--input	path to the file to compress
type	-t	--dtype	f32/f4, f64/f8
extent	-l	--len, --xyz	[x], [x]-[y], [x]-[y]-[z]
mode	-m	--mode	r2r (default), abs
bound	-e	--eb	e.g. 1e-4
predictor	-p,	--predictor	lrz (default), spl (spl-y25), spl-y24
1st codec	-c1	--codec1	Huffman: hfr-v4 (default), hf-rev2, hfr-v3, hfr-pbkc
			Others: lc, fzgcodec
2nd codec	-c2	--codec2	lc, rtr, bitr, hi-cr, hi-tp
preset	-s	--scheme	tp (codec1=lc), cr (codec1=hfr-v4)
report	-R	--report	cr
skip	-S	--skip	write2disk (no output to disk)

Decompression

• Subcommand form: cusz decompress or cusz unzip
• Command form cusz -x, or cusz --unzip

Part	Flag	Long form	Values
input	-i	--input	path to .cusza archive
verify		--compare	the original file; prints PSNR and max error
skip	-S	--skip	write2disk (no output to disk)

The report is enabled only on --compare presents.

Spline

cusz -t f32 -m r2r -e ${EB} -i ${CESM} -l 3600-1800 --predictor spl -z --report cr
cusz -i ${CESM}.cusza -x --compare ${CESM}

Two variants exist baased on research papers in differnt years:

Flag	Variant	Dims
-p spl, spl-y25	y25 (default), "cuSZ-Hi"	2D and 3D
-p spl-y24	y24, "cuSZ-i"	3D only

Spline is interpolation-based and generally reaches a higher ratio than Lorenzo at the same input eb on smooth fields, at a cost of lowered throughput currently. On a 3D field:

export HURR=/path/to/CLOUDf48.f4
cusz -t f32 -m r2r -e ${EB} -i ${HURR} -l 500-500-100 --predictor spl -z --report cr
cusz -i ${HURR}.cusza -x --compare ${HURR}

The LC path

Caution

Bugfix for LC path is in progress.

LC replaces the Huffman stage with the LC-framework codecs. Three shapes:

# TCMS-only: eq is TCMS-compressed, anchor and outliers stay raw
cusz -t f32 -m r2r -e ${EB} -i ${CESM} -l 3600-1800 -c1 lc -z --report cr
cusz -i ${CESM}.cusza -x --compare ${CESM}

# HiTP: TCMS over eq, BITR over anchor+outliers
cusz -t f32 -m r2r -e ${EB} -i ${CESM} -l 3600-1800 -c1 lc -c2 lc -z --report cr

# HiCR: Huffman over eq, RTR over the whole encoded block
cusz -t f32 -m r2r -e ${EB} -i ${CESM} -l 3600-1800 -c1 hf -c2 lc -z --report cr

--scheme tp is shorthand for --codec1 lc, so it takes the same path. --codec1 tcms is accepted as a synonym for lc.

LC trades ratio for throughput: on ${CESM} at -m r2r -e 1e-3, TCMS-only reaches about CR 3.2 where the default Huffman path reaches about 13.5, at the same PSNR. Whether that trade is worth it depends on whether the bitstream stage is your bottleneck.

The FZG path

cusz -t f32 -m r2r -e ${EB} -i ${CESM} -l 3600-1800 --predictor lrz --codec1 fzgcodec -z --report cr
cusz -i ${CESM}.cusza -x --compare ${CESM}

pSZ/cuSZ wires the codec in FZ-GPU (bitshuffle-based de-redundancy) and uses it as-is (unmodified). To enabled the FZGPU-codec (stock FZ-GPU alike), -c1 fzgcodec is needed.

Measured against stock (unmodified) FZ-GPU, same GPU, -m r2r -e 1e-3:

Dataset	Shape	cuSZ FZG (CR)	Stock FZ-GPU (CR)	cuSZ HF (CR)
CESM CLDHGH	3600-1800 (2D)	8.82	8.54	14.03
Hurricane CLOUDf48	500-500-100 (3D)	20.99	12.28	21.64
NYX baryon_density	512-512-512 (3D)	207.15	183.98	31.79

Note

Generally, the CR of pSZ/cuSZ+FZG-codec lands between the stock FZ-GPU and the Huffman path of pSZ/cuSZ: Hurricane +71%, NYX +12.5%, at a cost of lowered throughput due to the current wiring scheme: ~8-15% of compress and ~17-29% of decompress across the above three datasets at the tested eb.

Checking a run is actually lossless-to-bound

--compare <original> prints PSNR and max error. Two habits worth keeping:

• Read the sign. A broken decode reports a negative PSNR. A regular expression like PSNR=[0-9.]+ silently matches nothing on those lines and reads as "no output" rather than as failure.
• Use a bound tight enough to produce outliers. At a loose bound a field may quantize to zero out-of-radius values, so a path that drops outliers entirely still round-trips correctly and looks healthy. -m abs -e 1e-4 on ${CESM} exercises them; -m r2r -e 1e-3 largely does not.

Skipping stages

cusz ... -z --skip huffman        # predictor only, no entropy coding
cusz ... -x --skip write2disk     # decompress and report without writing output

Useful for isolating where time goes: run once whole, once with the codec skipped, and the difference is the codec's share.