[Klaud Cold] qwen3.5-fp8-mi300x-sglang-agentic-mtp: first Qwen3.5 MI300X AgentX recipe, MTP with golden-AL simulated acceptance / MI300X 首个 Qwen3.5 AgentX 配方（MTP + 黄金 AL 模拟接受）

[functionstackx]

Summary

Adds qwen3.5-fp8-mi300x-sglang-agentic-mtp — the first Qwen3.5 AgentX (agentic-coding) recipe on MI300X (gfx942), shipped spec-decode-only per the AgentX policy in MODELS.md (agentic arms run with speculative decoding enabled instead of an STP/MTP A/B). One new script, one master-config entry, one changelog entry; launch_mi300x-amds.sh already composes agentic/ + _mtp.

Speculative config — mirrors fixed_seq_len/qwen3.5_fp8_mi325x_mtp.sh, the other gfx942 Qwen3.5 MTP recipe: SGLANG_ENABLE_SPEC_V2=1, --speculative-algorithm EAGLE, --speculative-num-steps 3, --speculative-eagle-topk 1, --speculative-num-draft-tokens 4. MI300X had no Qwen3.5 MTP entry at all before this; MI325X is the same architecture and already runs this shape.

Synthetic forced AL — throughput runs export the SGLang triple from golden_al_distribution/README.md:

SGLANG_SIMULATE_ACC_LEN=3.39          # qwen3.5_mtp.yaml, thinking_on, K=3
SGLANG_SIMULATE_ACC_METHOD=match-expected
SGLANG_SIMULATE_ACC_TOKEN_MODE=real-draft-token

EVAL_ONLY leaves simulated acceptance off and keeps real verification.

Image lmsysorg/sglang-rocm:v0.5.16-rocm720-mi30x-20260730 (tag verified on Docker Hub). SGLANG_SIMULATE_ACC_TOKEN_MODE is only read from v0.5.16; the existing gfx942 Qwen3.5 entries pin v0.5.12-rocm720-mi30x, which would silently ignore that half of the contract.

Replay path follows the MI355X Qwen3.5 AgentX recipe rather than the fixed-seq-len gfx942 one: --attention-backend triton (fixed-seq-len ROCm recipes use aiter, but the agentic corpus is validated on triton), HiCache forced onto the page_size=1 direct/layer_first copy path — Qwen3.5's hybrid Mamba path runs SGLang's no_buffer scheduler and the kernel/page_first transfer path faults on first prefill on ROCm — plus --skip-server-warmup so aiperf owns benchmark traffic. --mem-fraction-static 0.75, as on fixed_seq_len/qwen3.5_fp8_mi300x.sh for this 192 GB SKU.

Search space covers the TP8 GPU-resident and HiCache arms on the AgentX MTP concurrency grid: GPU-resident rows [1, 4, 8, 12, 16], KV-offload rows [4, 8, 12, 16]. Every step is at least 2 concurrency apart and every arm stops hard at conc 16 — single-step sampling cannot separate configurations by more than run-to-run noise on the agentic corpus, and past 16 these SKUs are into the post-HBM-cliff thrashing regime the non-MTP sweeps already characterized. 9 generated jobs.

Validation: bash -n clean; both YAMLs parse; generate_sweep_configs.py test-config emits 9 agentic-coding configs with spec-decoding: mtp on cluster:mi300x-amds. First-on-SKU bring-up, so the bands and the triton/HiCache combination deserve a reviewer's eye before the sweep runs.

中文说明

新增 qwen3.5-fp8-mi300x-sglang-agentic-mtp —— MI300X（gfx942）上首个 Qwen3.5 AgentX（智能体编码）配方，按 MODELS.md 的 AgentX 规则仅发布投机解码版本（不再做 STP/MTP 对照）。包含一个新脚本、一条 master 配置条目与一条 changelog 条目；launch_mi300x-amds.sh 已能拼出 agentic/ + _mtp 路径。

投机解码配置与 fixed_seq_len/qwen3.5_fp8_mi325x_mtp.sh（另一条 gfx942 Qwen3.5 MTP 配方）一致：SGLANG_ENABLE_SPEC_V2=1、--speculative-algorithm EAGLE、--speculative-num-steps 3、--speculative-eagle-topk 1、--speculative-num-draft-tokens 4。MI300X 此前没有任何 Qwen3.5 MTP 条目；MI325X 属于同一架构且该配置已跑通。

合成强制 AL：吞吐量测试导出 golden_al_distribution/README.md 规定的三个 SGLang 环境变量：SGLANG_SIMULATE_ACC_LEN=3.39（qwen3.5_mtp.yaml，thinking_on，K=3）、SGLANG_SIMULATE_ACC_METHOD=match-expected、SGLANG_SIMULATE_ACC_TOKEN_MODE=real-draft-token。EVAL_ONLY 不启用模拟接受、保留真实验证。

镜像 lmsysorg/sglang-rocm:v0.5.16-rocm720-mi30x-20260730（tag 已在 Docker Hub 验证）。SGLANG_SIMULATE_ACC_TOKEN_MODE 自 v0.5.16 起才被读取；现有 gfx942 Qwen3.5 条目使用 v0.5.12-rocm720-mi30x，会静默忽略该约定。

回放路径沿用 MI355X 的 Qwen3.5 AgentX 配方而非固定序列长度的 gfx942 配方：--attention-backend triton（固定序列长度的 ROCm 配方用 aiter，但智能体语料是在 triton 上验证的）；HiCache 固定走 page_size=1 的 direct/layer_first 拷贝路径 —— Qwen3.5 的混合 Mamba 路径在 SGLang 上使用 no_buffer 调度器，而 ROCm 上 kernel/page_first 传输路径会在首次 prefill 时崩溃；并加上 --skip-server-warmup，由 aiperf 负责基准流量。--mem-fraction-static 取 0.75，与 fixed_seq_len/qwen3.5_fp8_mi300x.sh 在这块 192 GB SKU 上的设置一致。

搜索空间覆盖TP8 GPU 常驻与 HiCache 两条分支，并采用 AgentX MTP 并发网格：GPU 常驻 [1, 4, 8, 12, 16]，KV 卸载 [4, 8, 12, 16]。相邻并发间隔至少为 2，且一律止于并发 16 —— 在智能体语料上，逐 1 采样无法区分出超过运行间噪声的差异；而并发超过 16 后，这些 SKU 已进入非 MTP 扫描早已刻画过的显存瓶颈后抖动区间。共生成 9 个作业。

验证：bash -n 通过；两个 YAML 均可解析；generate_sweep_configs.py test-config 生成 9 条 cluster:mi300x-amds 上的 agentic-coding 配置。这是该 SKU 上的首次接入，建议评审确认并发区间与 triton/HiCache 组合后再启动扫描。

🤖 Generated with Claude Code

---

Revision

Concurrency grid. Steps of at least 2 and a hard stop at conc 16, replacing the sampling carried over from the non-MTP agentic shapes.

中文：并发网格调整为相邻间隔至少 2、一律止于并发 16，不再沿用非 MTP 智能体配方的采样方式。

Revision 3 — image swapped to the sglang-rocm build (Python 3.11)

The first job to actually start died ~30 s in, before serving:

uv pip install ... -e /workspace/utils/aiperf
Using Python 3.10.12 environment at: .../venv
x No solution found: the current Python version (3.10.12) does not satisfy Python>=3.11,<3.14
  and aiperf==0.12.0 depends on Python>=3.11,<3.14

lmsysorg/sglang:v0.5.16-rocm720-mi30x ships Python 3.10.12 in /opt/venv, and install_agentic_deps builds the AgentX venv from $(command -v python3), so uv can never resolve AIPerf. My earlier image bump verified the tag existed but not that it could run AgentX — that was the miss.

Now on lmsysorg/sglang-rocm:v0.5.16-rocm720-mi30x-20260730: same sglang 0.5.16 (so SGLANG_SIMULATE_ACC_TOKEN_MODE is still honored), same gfx942 arch, but the dated sglang-rocm family ships CPython 3.11.15 — confirmed from a green MI355X AgentX job running lmsysorg/sglang-rocm:v0.5.15.post1-rocm720-mi35x-20260719, which is the same build pipeline. Tag verified present on Docker Hub.

中文：首个真正启动的作业在约 30 秒后、尚未提供服务前即失败 —— lmsysorg/sglang:v0.5.16-rocm720-mi30x 的 /opt/venv 为 Python 3.10.12，而 install_agentic_deps 依据 $(command -v python3) 创建 AgentX 虚拟环境，uv 无法解析要求 Python>=3.11 的 AIPerf。此前的镜像升级只验证了 tag 存在，未验证其能运行 AgentX。现改用 lmsysorg/sglang-rocm:v0.5.16-rocm720-mi30x-20260730：sglang 仍为 0.5.16、架构仍为 gfx942，且该 dated 系列自带 CPython 3.11.15 —— 已由运行 lmsysorg/sglang-rocm:v0.5.15.post1-rocm720-mi35x-20260719 的成功 MI355X AgentX 作业确认，二者出自同一构建流程。Docker Hub 上已验证该 tag 存在。

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[functionstackx]

Golden-curve provenance note for reviewers. The pinned AL 3.39 comes from golden_al_distribution/qwen3.5_mtp.yaml, whose model key is qwen3.5-397b-a17b-nvfp4 — the curve was collected on the NVFP4 checkpoint on B300, while this recipe serves the FP8 checkpoint. There is no qwen3.5 FP8 curve committed in the repo, and the AgentX policy in golden_al_distribution/README.md keys the golden curve on model / thinking mode / draft length rather than on serving precision, so 3.39 is the only committed target available. The existing GB300 Qwen3.5 AgentX srt-slurm recipes pin the same 3.39, but they are FP4 — these are the first FP8 recipes to use that curve.

If the maintainers consider the curve precision-sensitive, the fix is a new SPEED-Bench collection on the FP8 checkpoint via speedbench-al.yml, not a hand-adjusted value here.

中文：本配方固定的 AL 3.39 取自 golden_al_distribution/qwen3.5_mtp.yaml，其模型键为 qwen3.5-397b-a17b-nvfp4 —— 该曲线是在 B300 上基于 NVFP4 权重测得的，而本配方服务的是 FP8 权重。仓库中没有提交 Qwen3.5 的 FP8 曲线，且 golden_al_distribution/README.md 的 AgentX 规则按「模型 / thinking 模式 / 草稿长度」而非推理精度来索引黄金曲线，因此 3.39 是目前唯一可用的既定目标值。现有 GB300 Qwen3.5 AgentX srt-slurm 配方同样固定为 3.39，但它们是 FP4；本批是首次将该曲线用于 FP8 配方。

若维护者认为该曲线对精度敏感，正确做法是通过 speedbench-al.yml 在 FP8 权重上重新采集一条 SPEED-Bench 曲线，而不是在此处手工调整数值。

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30516175647
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30516175647

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30517384501
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30517384501

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30518685035
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30518685035

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30518685035
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30518685035

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30518685035
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30518685035

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30570178838
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30570178838