[Klaud Cold] qwen3.5-fp8-h200-sglang-agentic-mtp: first Qwen3.5 H200 AgentX recipe, MTP with golden-AL simulated acceptance / H200 首个 Qwen3.5 AgentX 配方（MTP + 黄金 AL 模拟接受）

[functionstackx]

Summary

Adds qwen3.5-fp8-h200-sglang-agentic-mtp — the first Qwen3.5 AgentX (agentic-coding) recipe on H200. There is no non-MTP Qwen3.5 agentic arm on this SKU and this PR does not add one: per the AgentX policy in MODELS.md, agentic recipes are run and published with speculative decoding enabled rather than as an STP/MTP A/B, because synthetic acceptance already makes spec-decode results comparable across submissions. One new script, one master-config entry, one changelog entry; launch_h200-dgxc-slurm.sh already resolves agentic/…_mtp.sh.

Speculative config — mirrors fixed_seq_len/qwen3.5_fp8_h200_mtp.sh: SGLANG_ENABLE_SPEC_V2=1, --speculative-algorithm EAGLE, --speculative-num-steps 3, --speculative-eagle-topk 1, --speculative-num-draft-tokens 4 (3 speculative tokens per verification step).

Synthetic forced AL — throughput runs export the SGLang triple from golden_al_distribution/README.md:

SGLANG_SIMULATE_ACC_LEN=3.39          # qwen3.5_mtp.yaml, thinking_on, K=3
SGLANG_SIMULATE_ACC_METHOD=match-expected
SGLANG_SIMULATE_ACC_TOKEN_MODE=real-draft-token

Same golden AL the GB300 Qwen3.5 AgentX srt-slurm recipes pin. EVAL_ONLY leaves simulated acceptance off and keeps real verification.

Image lmsysorg/sglang:v0.5.16-cu130 — SGLANG_SIMULATE_ACC_TOKEN_MODE is only read from v0.5.16 (checked spec_utils.py at v0.5.12 / v0.5.14 / v0.5.16); the fixed-seq-len H200 Qwen3.5 entries run v0.5.14, so this is one minor version further, tag verified on Docker Hub.

Serve shape follows the proven H100 Qwen3.5 AgentX replay path (HiCache host-DRAM offload with page-size 64 / kernel IO / page_first, flashinfer attention, --quantization fp8, --kv-cache-dtype fp8_e4m3, --mamba-ssm-dtype bfloat16, the multi_tokenizer cached_tokens_details patch, the 256k-capped with-subagents trace corpus), with --mem-fraction-static 0.8 instead of 0.75 for H200's 141 GB HBM3e — matching the fixed-seq-len H200 recipe.

Search space covers the TP8/EP8 GPU-resident and HiCache arms on the AgentX MTP concurrency grid: GPU-resident rows [1, 4, 8, 12, 16], KV-offload rows [4, 8, 12, 16]. Every step is at least 2 concurrency apart and every arm stops hard at conc 16 — single-step sampling cannot separate configurations by more than run-to-run noise on the agentic corpus, and past 16 these SKUs are into the post-HBM-cliff thrashing regime the non-MTP sweeps already characterized. 9 generated jobs.

Validation: bash -n clean; both YAMLs parse; generate_sweep_configs.py test-config emits 9 agentic-coding configs with spec-decoding: mtp on cluster:h200-dgxc. Since this is a first-on-SKU bring-up, the concurrency bands are an estimate from the H100 agentic curve and the H200 fixed-seq-len ceiling — worth a look before the sweep burns GPU time.

中文说明

新增 qwen3.5-fp8-h200-sglang-agentic-mtp —— H200 上首个 Qwen3.5 AgentX（智能体编码）配方。该 SKU 上没有非 MTP 的 Qwen3.5 智能体分支，本 PR 也不新增：按 MODELS.md 的 AgentX 规则，智能体配方一律在启用投机解码的条件下运行与发布，不再做 STP/MTP 对照，因为合成接受已使各家投机解码结果可直接比较。包含一个新脚本、一条 master 配置条目与一条 changelog 条目；launch_h200-dgxc-slurm.sh 已能解析 agentic/…_mtp.sh。

投机解码配置与 fixed_seq_len/qwen3.5_fp8_h200_mtp.sh 一致：SGLANG_ENABLE_SPEC_V2=1、--speculative-algorithm EAGLE、--speculative-num-steps 3、--speculative-eagle-topk 1、--speculative-num-draft-tokens 4（每个验证步 3 个投机 token）。

合成强制 AL：吞吐量测试导出 golden_al_distribution/README.md 规定的三个 SGLang 环境变量：SGLANG_SIMULATE_ACC_LEN=3.39（qwen3.5_mtp.yaml，thinking_on，K=3）、SGLANG_SIMULATE_ACC_METHOD=match-expected、SGLANG_SIMULATE_ACC_TOKEN_MODE=real-draft-token，与 GB300 Qwen3.5 AgentX srt-slurm 配方所用数值相同。EVAL_ONLY 不启用模拟接受、保留真实验证。

镜像 lmsysorg/sglang:v0.5.16-cu130：SGLANG_SIMULATE_ACC_TOKEN_MODE 自 v0.5.16 起才被读取（已核对 v0.5.12 / v0.5.14 / v0.5.16 的 spec_utils.py）；固定序列长度的 H200 Qwen3.5 条目使用 v0.5.14，因此仅前进一个小版本，tag 已在 Docker Hub 验证。

服务参数沿用已验证的 H100 Qwen3.5 AgentX 回放路径（HiCache 主机 DRAM 卸载，page-size 64 / kernel IO / page_first，flashinfer 注意力后端，--quantization fp8、--kv-cache-dtype fp8_e4m3、--mamba-ssm-dtype bfloat16、multi_tokenizer 的 cached_tokens_details 补丁、256k 上限的 with-subagents 轨迹集），并针对 H200 的 141 GB HBM3e 将 --mem-fraction-static 从 0.75 调整为 0.8，与固定序列长度的 H200 配方一致。

搜索空间覆盖TP8/EP8 GPU 常驻与 HiCache 两条分支，并采用 AgentX MTP 并发网格：GPU 常驻 [1, 4, 8, 12, 16]，KV 卸载 [4, 8, 12, 16]。相邻并发间隔至少为 2，且一律止于并发 16 —— 在智能体语料上，逐 1 采样无法区分出超过运行间噪声的差异；而并发超过 16 后，这些 SKU 已进入非 MTP 扫描早已刻画过的显存瓶颈后抖动区间。共生成 9 个作业。

验证：bash -n 通过；两个 YAML 均可解析；generate_sweep_configs.py test-config 生成 9 条 cluster:h200-dgxc 上的 agentic-coding 配置。由于这是该 SKU 上的首次接入，并发区间是依据 H100 智能体曲线与 H200 固定序列上限的估计值，建议在扫描消耗 GPU 前先行确认。

🤖 Generated with Claude Code

---

Revision

Concurrency grid. Steps of at least 2 and a hard stop at conc 16, replacing the sampling carried over from the non-MTP agentic shapes.

中文：并发网格调整为相邻间隔至少 2、一律止于并发 16，不再沿用非 MTP 智能体配方的采样方式。

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[functionstackx]

Golden-curve provenance note for reviewers. The pinned AL 3.39 comes from golden_al_distribution/qwen3.5_mtp.yaml, whose model key is qwen3.5-397b-a17b-nvfp4 — the curve was collected on the NVFP4 checkpoint on B300, while this recipe serves the FP8 checkpoint. There is no qwen3.5 FP8 curve committed in the repo, and the AgentX policy in golden_al_distribution/README.md keys the golden curve on model / thinking mode / draft length rather than on serving precision, so 3.39 is the only committed target available. The existing GB300 Qwen3.5 AgentX srt-slurm recipes pin the same 3.39, but they are FP4 — these are the first FP8 recipes to use that curve.

If the maintainers consider the curve precision-sensitive, the fix is a new SPEED-Bench collection on the FP8 checkpoint via speedbench-al.yml, not a hand-adjusted value here.

中文：本配方固定的 AL 3.39 取自 golden_al_distribution/qwen3.5_mtp.yaml，其模型键为 qwen3.5-397b-a17b-nvfp4 —— 该曲线是在 B300 上基于 NVFP4 权重测得的，而本配方服务的是 FP8 权重。仓库中没有提交 Qwen3.5 的 FP8 曲线，且 golden_al_distribution/README.md 的 AgentX 规则按「模型 / thinking 模式 / 草稿长度」而非推理精度来索引黄金曲线，因此 3.39 是目前唯一可用的既定目标值。现有 GB300 Qwen3.5 AgentX srt-slurm 配方同样固定为 3.39，但它们是 FP4；本批是首次将该曲线用于 FP8 配方。

若维护者认为该曲线对精度敏感，正确做法是通过 speedbench-al.yml 在 FP8 权重上重新采集一条 SPEED-Bench 曲线，而不是在此处手工调整数值。

[claude]

LGTM — clean recipe addition following the established H100 agentic / H200 fixed-seq-len MTP patterns.

What was reviewed: the new benchmark script's diff against agentic/qwen3.5_fp8_h100.sh (only expected deltas: mem-fraction-static 0.8, EAGLE spec-decoding args, golden-AL simulated-acceptance env block gated on EVAL_ONLY, image bump to v0.5.16-cu130); the master-config search-space entry and changelog addition for consistency with the description. The flagged duplicate --served-model-name flag was ruled out (harmless, second occurrence wins) and the changelog pr-link is correctly populated (not a placeholder).

Extended reasoning...

Overview

This PR adds a single new agentic (agentic-coding) benchmark recipe, qwen3.5-fp8-h200-sglang-agentic-mtp, touching exactly three files: a new shell script (benchmarks/single_node/agentic/qwen3.5_fp8_h200_mtp.sh), one master-config entry (configs/nvidia-master.yaml), and one changelog entry (perf-changelog.yaml). No existing recipes, shared libraries, or launcher code are modified.

Security risks

None. This is benchmark/ops tooling that launches an SGLang server with fixed, non-user-controlled arguments and downloads a model via hf download. No secrets, auth, or externally-reachable surface are touched. The in-place sed patch to the vendored multi_tokenizer_mixin.py inside the container is copied verbatim from the existing proven H100 sibling script and is idempotent (guarded by a grep check).

Level of scrutiny

Low-to-medium is appropriate: this is a net-new, isolated recipe addition (not a modification of shared/critical logic), and the diff against its two template siblings (agentic/qwen3.5_fp8_h100.sh and fixed_seq_len/qwen3.5_fp8_h200_mtp.sh) shows only the deltas the PR description claims — mem-fraction-static, EAGLE spec-decoding block, golden-AL env vars gated on EVAL_ONLY, and the image tag bump. The bug-hunting system found no issues, and the two candidate issues it ruled out (duplicate --served-model-name, changelog pr-link placeholder) are non-problems on inspection of the diff.

Other factors

This is a bring-up recipe explicitly labeled as an estimate ("concurrency bands are an estimate ... worth a look before the sweep burns GPU time") and gated behind full-sweep-fail-fast, so correctness will be exercised by CI before merge. No outstanding reviewer comments exist in the timeline beyond the standard automated CODEOWNER reminder.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30516130371
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30516130371

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30517383054
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30517383054

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30517383054
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30517383054