HyperWafer: Communication-Aware Sparse Matrix Multiplication on Wafer-Scale Chips

HyperWafer is an efficient Hypergraph-based SpGEMM Partitioning system designed for Wafer-scale or large-scale mesh architectures.

The goal of HyperWafer is to accelerate sparse matrix-matrix multiplication (SpGEMM) by minimizing communication volume and peak link traffic. It integrates Gustavson-based hypergraph construction, mesh-aware partitioning (via Mt-KaHyPar), and a high-fidelity communication oracle for 2D mesh topologies.

---

🚀 Key Features

• Gustavson-based Hypergraph Modeling: Captures SpGEMM row-wise product dependencies accurately.
• Mesh-aware Partitioning: Leverages NoC topology during partitioning to minimize GB-hop and link congestion.
• 2D Mesh Oracle: Precisely simulates multicast/broadcast communication patterns on mesh networks.
• End-to-End Simulation Pipeline: Integrates with Chakra and Astra-Sim for performance profiling and cycle-level analysis.
• Support for Real-world Workloads: Direct interface with SuiteSparse matrix collection.

---

📁 Directory Structure

HyperWafer-AE/
├── src/            # Core C++ implementation and Python pipeline scripts
├── include/        # Public C++ headers
├── script/         # Automation and Astra-Sim build scripts
├── hyperwafer-sim/ # Simulation backend and logic (System, Network, Workload)
├── examples/       # Astra-Sim configuration templates
├── extern/         # External dependencies and submodules
├── CMakeLists.txt  # Project build configuration
└── README.md       # Project description

---

🛠 Requirements

• Linux
• C++17 compiler
• CMake 3.22 or newer
• Mt-KaHyPar binary
• Chakra chakra_converter
• AstraSim analytical congestion-aware binary

The C++ pipeline does not use Python, SciPy, NumPy, ssgetpy, or SuiteSparse download helpers. Provide a local MatrixMarket .mtx file as input.

⚙️ Run

./build/bin/hyperwafer_cli \
  --matrix path/to/A.mtx \
  --num-parts 256 \
  --mtk-bin /path/to/MtKaHyPar \
  --chakra-bin /path/to/chakra_converter \
  --astrasim-bin /path/to/AstraSim_Analytical_Congestion_Aware \
  --workdir ./runs/example

By default, B = A^T. The default AstraSim configs are:

• examples/config/system_default.json
• examples/config/remote_memory_none.json

If --network-config auto is used, the CLI writes a Mesh2D network config to the work directory using the same factorization as the internal 2D mesh oracle.

📈 Outputs

The work directory contains:

• matrix_info.txt
• hypergraph_weighted.hgr
• noc_<num_parts>n_mesh.metis
• network_mesh_<num_parts>.yml when --network-config auto
• mtk_parts/<partition-file>
• SpGEMM_WaferSpMM.txt
• SpGEMM_HyperWafer.txt
• SpGEMM_WaferSpMM.*.et
• SpGEMM_HyperWafer.*.et
• summary.json

The terminal summary reports total broadcast bytes, GB-hop, peak link traffic, MICRO layers, AstraSim communication time, and reduction ratios.

Astra-Sim Integration

HyperWafer outputs Chakra-compatible execution traces that can be further analyzed using Astra-Sim's profiling tools to visualize workload distribution and link utilization across the wafer mesh.

License

MIT License