feat(pytorch): add optimized Gluon blocked FP8 GEMM for Hopper

[grimoire]

Kernel design

The kernel uses three shape-specific schedules:

• M <= 128: single-partition multistage kernel, with a transposed GEMM schedule for M <= 8 and large K;
• 129 <= M <= 256: non-persistent, warp-specialized pipeline;
• M > 256: persistent, warp-specialized kernel using two M waves and scale caching.

Dispatch

The automatic provider order is:

1. DeepGEMM
2. Gluon
3. Triton fallback

The provider can be overridden with:

LMDEPLOY_BLOCKED_FP8_GEMM_BACKEND={auto,deepgemm,gluon,triton}

Performance

Measured on an NVIDIA H200 with N=K=4096, identical preallocated inputs, 50 ms warmup, 250 ms measurement, and five alternating trials. Values are median p50 latency.

M	Triton µs	Gluon µs	DeepGEMM µs	Gluon TFLOP/s	Gluon/Triton	Gluon/DeepGEMM
1	24.30	13.42	14.94	2.50	0.55x	0.90x
8	24.83	14.21	12.58	18.89	0.57x	1.13x
9	25.12	14.24	12.58	21.21	0.57x	1.13x
16	26.27	14.69	12.64	36.55	0.56x	1.16x
24	26.26	14.08	12.64	57.20	0.54x	1.11x
32	26.50	15.09	12.83	71.17	0.57x	1.18x
64	26.94	14.58	12.56	147.33	0.54x	1.16x
128	28.00	16.10	13.15	266.83	0.57x	1.22x
160	28.51	19.12	14.14	280.79	0.67x	1.35x
192	28.21	16.72	14.19	385.31	0.59x	1.18x
224	28.93	19.22	15.30	391.14	0.66x	1.26x
256	28.98	17.52	15.36	490.29	0.60x	1.14x
512	31.28	21.23	21.18	809.15	0.68x	1.00x
1024	56.45	34.30	32.48	1001.62	0.61x	1.06x
2048	90.30	60.98	61.87	1126.99	0.68x	0.99x
4096	181.04	125.68	127.12	1093.56	0.69x	0.99x
8192	355.09	255.46	253.40	1076.03	0.72x	1.01x

Warning

The kernel for large M follow the same design of deepgemm, but kernel for middle/small M failed to get the same prerformance.

[Copilot]

Pull request overview

This PR introduces a new Hopper-targeted, Gluon/WGMMA-based blocked FP8 GEMM path for LMDeploy’s PyTorch backend, integrates it into the blocked-FP8 Linear provider dispatch, and adds correctness/determinism tests plus supporting tooling and environment/config updates.

Changes:

• Add a shape-specialized Gluon blocked FP8 GEMM kernel for SM90 (Hopper) with dispatch across small/mid/persistent schedules.
• Extend CUDA blocked-FP8 Linear provider selection (DeepGEMM → Gluon → Triton) and add LMDEPLOY_BLOCKED_FP8_GEMM_BACKEND override.
• Add Hopper/Gluon kernel correctness + determinism tests and introduce a dependency-version auditing maintenance script; update FP8 scale alignment for Hopper TMA.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/pytorch/kernel/test_blocked_fp8_gemm_gluon.py	Adds Hopper/Gluon blocked-FP8 GEMM correctness, determinism, and contract-validation tests (skipped when unsupported).
lmdeploy/pytorch/tools/audit_dependency_versions.py	Adds a read-only tool to inventory dependency requirements and source-level version checks.
lmdeploy/pytorch/nn/linear/blocked_fp8.py	Plumbs block_size/fp8_dtype into the blocked-FP8 Linear builder call.
lmdeploy/pytorch/kernels/cuda/blocked_gemm_fp8.py	Pads FP32 scale strides to satisfy Hopper TMA 16-byte alignment requirements.
lmdeploy/pytorch/kernels/cuda/blocked_fp8_gemm_gluon.py	Introduces the new Gluon blocked FP8 GEMM implementation with schedule dispatch and SM budgeting support.
lmdeploy/pytorch/envs.py	Adds LMDEPLOY_BLOCKED_FP8_GEMM_BACKEND env parsing for provider selection.
lmdeploy/pytorch/check_env/transformers.py	Replaces min/max version checks with a SpecifierSet-based Transformers version policy warning.
lmdeploy/pytorch/backends/cuda/op_backend.py	Switches blocked-FP8 Linear builder to the new CUDA provider-dispatch builder.
lmdeploy/pytorch/backends/cuda/blockedf8_modules.py	Refactors blocked-FP8 Linear implementations into a common CUDA base and adds Gluon provider + provider selection logic.
lmdeploy/pytorch/backends/blockedf8_modules.py	Extends the blocked-FP8 Linear builder interface to accept block_size and fp8_dtype.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.