ISC-Parquet-Page-Pruning

Page-pruning of Parquet page indexes, evaluated on three platforms: Host CPU, ARM, and an FPGA (HLS). This repository reproduces the experiments in the paper The In-Storage Computing Trilemma and How to Sidestep It: Parquet Page Pruning Offload.

Repository layout

Directory	Contents
Host_CPU/	Rust code to generate the synthetic Parquet datasets and their metadata
ARM/	Rust metadata-parsing code, cross-compiled for ARM; also run on the host CPU for the processing-time measurements
FPGA_HLS/	High-level-synthesis sources for the FPGA Kernel
graphs/	Scripts and data to produce the paper's figures from the results

Running the experiments

Host CPU

Generates the synthetic Parquet files, then builds the FlatBuffers-based metadata for them. As input it needs a table from the SSB benchmark (the schema is provided under gen_data_set/); this example uses the lineorder table, which you place in the input_tbl/ directory, where the Python script expects to find it (The table must be a .tbl file with | as the column separator which is the standard SSB format). This code is part of a larger project, so the required Cargo features must be enabled when running it, as shown below.

cd Host_CPU
python3 gen_data_set/gen_dist.py lineorder
cargo run --features "parq_generator multi_run" -- lineorder > out.log

ARM

The code reads FlatBuffers-based input files from input_fb_files/, which is already populated; alternatively, copy the files produced by the Host_CPU/gen_metadata/ experiment into input_fb_files/ and run against those. The measured statistics are written to out_stats/. This runs on the ARM processor; the same code is also used to measure metadata parsing on the host CPU.

cd ARM/meta_parsing/parsing
cargo run input_fb_files/

FPGA (HLS)

Requires the AMD Vitis HLS tools to be installed. This runs C simulation against a testbench, which checks the kernel's output against the expected result, then runs synthesis followed by implementation. The run produces several files in the project directory; the results we care about are in solution1/impl/report/verilog/export_impl.xml (implementation) and solution1/syn/report/csynth.xml (synthesis).

cd FPGA_HLS
vitis_hls -f run_hls.tcl

Reproducing the figures

Sample inputs for the graphs are already provided in the input_files directory, so the figures can be regenerated directly. To plot your own runs instead, copy the CSV files generated in the out_stats directory (under the ARM/ experiment) into input_files/ here, replacing ARM.csv (the ARM run) and host.csv (the Host CPU run). The FPGA resource numbers must be read from export_impl.xml.

cd graphs
python3 latency_draw.py