[Klaud Cold] qwen3.5-fp8-h100-sglang-agentic-mtp: Qwen3.5 H100 AgentX MTP recipe with golden-AL simulated acceptance / Qwen3.5 H100 AgentX MTP 配方（黄金 AL 模拟接受）

[functionstackx]

Summary

Adds qwen3.5-fp8-h100-sglang-agentic-mtp — the MTP speculative-decoding (spec-decoding: mtp) variant of the Qwen3.5-397B-A17B FP8 H100 AgentX recipe. One new benchmark script, one master-config entry, one changelog entry; launch_h100-dgxc-slurm.sh already composes agentic/ + _mtp, so no launcher change.

Speculative config — mirrors fixed_seq_len/qwen3.5_fp8_h100_mtp.sh: SGLANG_ENABLE_SPEC_V2=1 with --speculative-algorithm EAGLE, --speculative-num-steps 3, --speculative-eagle-topk 1, --speculative-num-draft-tokens 4 — 3 speculative tokens per verification step.

Synthetic forced AL — throughput runs export the exact triple golden_al_distribution/README.md prescribes for SGLang:

SGLANG_SIMULATE_ACC_LEN=3.39          # qwen3.5_mtp.yaml, thinking_on, K=3
SGLANG_SIMULATE_ACC_METHOD=match-expected
SGLANG_SIMULATE_ACC_TOKEN_MODE=real-draft-token

3.39 is the same golden AL the GB300 Qwen3.5 AgentX srt-slurm recipes pin. EVAL_ONLY runs leave simulated acceptance off and keep real verification — committing drafted tokens regardless of the target logits would score the eval at ~0.

Image bump, deliberately — from the non-MTP agentic sibling's lmsysorg/sglang:v0.5.12-cu130 to v0.5.16-cu130. SGLANG_SIMULATE_ACC_TOKEN_MODE is only read from v0.5.16 (SIMULATE_ACC_LEN / _METHOD go back further), so an older image would silently ignore half the AgentX contract — I checked python/sglang/srt/speculative/spec_utils.py at v0.5.12 / v0.5.14 / v0.5.16 to confirm. v0.5.14 is already what the fixed-seq-len H100 Qwen3.5 MTP entry runs, so this is one minor version beyond an exercised build, and the tag is verified present on Docker Hub.

Serve shape is otherwise identical to the non-MTP agentic sibling (HiCache host-DRAM offload with page-size 64 / kernel IO / page_first layout, flashinfer attention on sm_90, --quantization fp8, --kv-cache-dtype fp8_e4m3, --mamba-ssm-dtype bfloat16, --mem-fraction-static 0.75, the multi_tokenizer cached_tokens_details patch).

Search space covers the TP8/EP8 GPU-resident and HiCache arms on the AgentX MTP concurrency grid: GPU-resident rows [1, 4, 8, 12, 16], KV-offload rows [4, 8, 12, 16]. Every step is at least 2 concurrency apart and every arm stops hard at conc 16 — single-step sampling cannot separate configurations by more than run-to-run noise on the agentic corpus, and past 16 these SKUs are into the post-HBM-cliff thrashing regime the non-MTP sweeps already characterized. 9 generated jobs.

Validation: bash -n clean; both YAMLs parse; generate_sweep_configs.py test-config emits 9 agentic-coding configs with spec-decoding: mtp on cluster:h100-dgxc.

中文说明

新增 qwen3.5-fp8-h100-sglang-agentic-mtp：Qwen3.5-397B-A17B FP8 H100 AgentX（智能体编码）配方的 MTP 投机解码（spec-decoding: mtp）变体。包含一个新脚本、一条 master 配置条目与一条 changelog 条目；launch_h100-dgxc-slurm.sh 已能拼出 agentic/ + _mtp 路径，启动器无需改动。

投机解码配置与 fixed_seq_len/qwen3.5_fp8_h100_mtp.sh 保持一致：SGLANG_ENABLE_SPEC_V2=1，--speculative-algorithm EAGLE、--speculative-num-steps 3、--speculative-eagle-topk 1、--speculative-num-draft-tokens 4，即每个验证步 3 个投机 token。

合成强制 AL：吞吐量测试导出 golden_al_distribution/README.md 为 SGLang 规定的三个环境变量：SGLANG_SIMULATE_ACC_LEN=3.39（qwen3.5_mtp.yaml，thinking_on，K=3）、SGLANG_SIMULATE_ACC_METHOD=match-expected、SGLANG_SIMULATE_ACC_TOKEN_MODE=real-draft-token。3.39 与 GB300 Qwen3.5 AgentX srt-slurm 配方所用数值一致。EVAL_ONLY 不启用模拟接受、保留真实验证，否则评估分数接近 0。

镜像升级（有意为之）：从非 MTP 版本的 lmsysorg/sglang:v0.5.12-cu130 升到 v0.5.16-cu130。SGLANG_SIMULATE_ACC_TOKEN_MODE 自 v0.5.16 起才被读取（SIMULATE_ACC_LEN / _METHOD 更早就有），旧镜像会静默忽略这一半约定 —— 已核对 v0.5.12 / v0.5.14 / v0.5.16 的 python/sglang/srt/speculative/spec_utils.py。固定序列长度的 H100 Qwen3.5 MTP 条目已在使用 v0.5.14，因此这只是再前进一个小版本，且该 tag 已在 Docker Hub 验证存在。

服务参数与非 MTP 版本一致（HiCache 主机 DRAM 卸载，page-size 64 / kernel IO / page_first 布局，sm_90 上的 flashinfer 注意力后端，--quantization fp8、--kv-cache-dtype fp8_e4m3、--mamba-ssm-dtype bfloat16、--mem-fraction-static 0.75，以及 multi_tokenizer 的 cached_tokens_details 补丁）。

搜索空间覆盖TP8/EP8 GPU 常驻与 HiCache 两条分支，并采用 AgentX MTP 并发网格：GPU 常驻 [1, 4, 8, 12, 16]，KV 卸载 [4, 8, 12, 16]。相邻并发间隔至少为 2，且一律止于并发 16 —— 在智能体语料上，逐 1 采样无法区分出超过运行间噪声的差异；而并发超过 16 后，这些 SKU 已进入非 MTP 扫描早已刻画过的显存瓶颈后抖动区间。共生成 9 个作业。

验证：bash -n 通过；两个 YAML 均可解析；generate_sweep_configs.py test-config 生成 9 条 cluster:h100-dgxc 上的 agentic-coding 配置。

🤖 Generated with Claude Code

---

Revision

Concurrency grid. Steps of at least 2 and a hard stop at conc 16, replacing the sampling carried over from the non-MTP agentic shapes.

中文：并发网格调整为相邻间隔至少 2、一律止于并发 16，不再沿用非 MTP 智能体配方的采样方式。

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[functionstackx]

Golden-curve provenance note for reviewers. The pinned AL 3.39 comes from golden_al_distribution/qwen3.5_mtp.yaml, whose model key is qwen3.5-397b-a17b-nvfp4 — the curve was collected on the NVFP4 checkpoint on B300, while this recipe serves the FP8 checkpoint. There is no qwen3.5 FP8 curve committed in the repo, and the AgentX policy in golden_al_distribution/README.md keys the golden curve on model / thinking mode / draft length rather than on serving precision, so 3.39 is the only committed target available. The existing GB300 Qwen3.5 AgentX srt-slurm recipes pin the same 3.39, but they are FP4 — these are the first FP8 recipes to use that curve.

If the maintainers consider the curve precision-sensitive, the fix is a new SPEED-Bench collection on the FP8 checkpoint via speedbench-al.yml, not a hand-adjusted value here.

中文：本配方固定的 AL 3.39 取自 golden_al_distribution/qwen3.5_mtp.yaml，其模型键为 qwen3.5-397b-a17b-nvfp4 —— 该曲线是在 B300 上基于 NVFP4 权重测得的，而本配方服务的是 FP8 权重。仓库中没有提交 Qwen3.5 的 FP8 曲线，且 golden_al_distribution/README.md 的 AgentX 规则按「模型 / thinking 模式 / 草稿长度」而非推理精度来索引黄金曲线，因此 3.39 是目前唯一可用的既定目标值。现有 GB300 Qwen3.5 AgentX srt-slurm 配方同样固定为 3.39，但它们是 FP4；本批是首次将该曲线用于 FP8 配方。

若维护者认为该曲线对精度敏感，正确做法是通过 speedbench-al.yml 在 FP8 权重上重新采集一条 SPEED-Bench 曲线，而不是在此处手工调整数值。

[claude]

LGTM — straightforward MTP variant recipe addition following an established pattern.

What was reviewed: diffed the new script against its non-MTP agentic sibling (qwen3.5_fp8_h100.sh) and confirmed the delta is exactly the speculative-decoding block (EAGLE args + simulated-acceptance env vars), matching the fixed_seq_len MTP sibling's shape. Verified the golden AL value (3.39, thinking_on, K=3) against golden_al_distribution/qwen3.5_mtp.yaml, and confirmed launch_h100-dgxc-slurm.sh already routes spec-decoding: mtp configs to the _mtp script suffix, so no launcher change was needed here.

Extended reasoning...

Overview

This PR adds one new benchmark script (benchmarks/single_node/agentic/qwen3.5_fp8_h100_mtp.sh), one master-config entry, and one perf-changelog entry for the MTP speculative-decoding variant of an existing Qwen3.5 FP8 H100 AgentX recipe. No launcher or shared-library changes.

Security risks

None. This is a benchmark recipe (shell script invoking sglang.launch_server with fixed CLI flags, config YAML, changelog text) — no user input, auth, or credential handling involved. The one dynamic piece is a sed-based idempotent patch to a vendored sglang file inside the container image, which is copy-pasted unchanged from the existing non-MTP sibling.

Level of scrutiny

Low-to-moderate is appropriate: this is an additive, mechanical recipe following an already-exercised pattern (the same MTP speculative-decoding shape as fixed_seq_len/qwen3.5_fp8_h100_mtp.sh, and the same golden-AL simulated-acceptance contract as the GB300 AgentX srt-slurm recipes). I diffed the new script against its non-MTP agentic sibling and the delta is exactly the speculative-decoding/simulated-acceptance block plus the image-version bump — no unrelated changes snuck in. The image bump to v0.5.16 is justified in-repo (v0.5.14 already runs in the fixed-seq-len MTP entry) and the bug-hunting system's own investigation ruled out concerns about it.

Other factors

The bug hunting system found no bugs, and its three candidate issues (image-bump risk, changelog TBD placeholder, HiCache pool-count budget) were all examined and refuted. I independently verified the golden AL value and the launcher's existing _mtp routing hold up against current repo state. The PR carries the standard full-sweep-fail-fast label so CI will validate the actual sweep; no CODEOWNER or human reviewer comments are outstanding on the timeline.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30516060125
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30516060125

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30517381812
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30517381812

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30517381812
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30517381812