LLM speculative inference server for consumer & heterogeneous hardware
An agent harness that compiles a model into one provably-correct, self-retargeting CUDA megakernel and self-tunes it past cuBLAS at batch-1 LLM decode, paper: https://arxiv.org/abs/2606.09682
Single-launch batch-1 decode engine for PrismML Bonsai 27B (ternary and 1-bit) on NVIDIA GPUs. 1.76x the vendor llama.cpp fork on H100, same outputs.
A light, transparent, and modular inference & quantization engine for studying LLMs.
Persistent-kernel CPU→GPU command bus. 64-byte packets, 0.5 µs enqueue, ~96 ns dispatch — no cudaLaunchKernel. C++/CUDA, 2 files.
Vendor-portable GPU decoders for quantum LDPC codes — Triton min-sum BP & Relay-BP on NVIDIA (CUDA), AMD (ROCm), and Apple-silicon (Metal), consuming any stim DetectorErrorModel.
Add a description, image, and links to the megakernel topic page so that developers can more easily learn about it.
To associate your repository with the megakernel topic, visit your repo's landing page and select "manage topics."