[Klaud Cold] qwen3.5-fp8-mi325x-sglang-agentic-mtp: first Qwen3.5 MI325X AgentX recipe, MTP with golden-AL simulated acceptance / MI325X 首个 Qwen3.5 AgentX 配方（MTP + 黄金 AL 模拟接受）

[functionstackx]

Summary

Adds qwen3.5-fp8-mi325x-sglang-agentic-mtp — the first Qwen3.5 AgentX (agentic-coding) recipe on MI325X (gfx942), shipped spec-decode-only per the AgentX policy in MODELS.md (agentic arms run with speculative decoding enabled instead of an STP/MTP A/B). One new script, one master-config entry, one changelog entry; launch_mi325x-amds.sh already composes agentic/ + _mtp.

Speculative config — mirrors fixed_seq_len/qwen3.5_fp8_mi325x_mtp.sh, the other gfx942 Qwen3.5 MTP recipe: SGLANG_ENABLE_SPEC_V2=1, --speculative-algorithm EAGLE, --speculative-num-steps 3, --speculative-eagle-topk 1, --speculative-num-draft-tokens 4. MI325X already runs this exact MTP shape in fixed_seq_len/qwen3.5_fp8_mi325x_mtp.sh; what is new here is the agentic-coding scenario on that SKU.

Synthetic forced AL — throughput runs export the SGLang triple from golden_al_distribution/README.md:

SGLANG_SIMULATE_ACC_LEN=3.39          # qwen3.5_mtp.yaml, thinking_on, K=3
SGLANG_SIMULATE_ACC_METHOD=match-expected
SGLANG_SIMULATE_ACC_TOKEN_MODE=real-draft-token

EVAL_ONLY leaves simulated acceptance off and keeps real verification.

Image lmsysorg/sglang-rocm:v0.5.16-rocm720-mi30x-20260730 (tag verified on Docker Hub). SGLANG_SIMULATE_ACC_TOKEN_MODE is only read from v0.5.16; the fixed-seq-len MI325X entry pins v0.5.12-rocm720-mi30x, which would silently ignore that half of the contract.

Replay path follows the MI355X Qwen3.5 AgentX recipe rather than the fixed-seq-len gfx942 one: --attention-backend triton (fixed-seq-len ROCm recipes use aiter, but the agentic corpus is validated on triton), HiCache forced onto the page_size=1 direct/layer_first copy path — Qwen3.5's hybrid Mamba path runs SGLang's no_buffer scheduler and the kernel/page_first transfer path faults on first prefill on ROCm — plus --skip-server-warmup so aiperf owns benchmark traffic. --mem-fraction-static 0.75, as on fixed_seq_len/qwen3.5_fp8_mi325x.sh for this 256 GB SKU.

Search space covers the TP8 GPU-resident and HiCache arms on the AgentX MTP concurrency grid: GPU-resident rows [1, 4, 8, 12, 16], KV-offload rows [4, 8, 12, 16]. Every step is at least 2 concurrency apart and every arm stops hard at conc 16 — single-step sampling cannot separate configurations by more than run-to-run noise on the agentic corpus, and past 16 these SKUs are into the post-HBM-cliff thrashing regime the non-MTP sweeps already characterized. 9 generated jobs.

Validation: bash -n clean; both YAMLs parse; generate_sweep_configs.py test-config emits 9 agentic-coding configs with spec-decoding: mtp on cluster:mi325x-amds. First-on-SKU bring-up, so the bands and the triton/HiCache combination deserve a reviewer's eye before the sweep runs.

中文说明

新增 qwen3.5-fp8-mi325x-sglang-agentic-mtp —— MI325X（gfx942）上首个 Qwen3.5 AgentX（智能体编码）配方，按 MODELS.md 的 AgentX 规则仅发布投机解码版本（不再做 STP/MTP 对照）。包含一个新脚本、一条 master 配置条目与一条 changelog 条目；launch_mi325x-amds.sh 已能拼出 agentic/ + _mtp 路径。

投机解码配置与 fixed_seq_len/qwen3.5_fp8_mi325x_mtp.sh（另一条 gfx942 Qwen3.5 MTP 配方）一致：SGLANG_ENABLE_SPEC_V2=1、--speculative-algorithm EAGLE、--speculative-num-steps 3、--speculative-eagle-topk 1、--speculative-num-draft-tokens 4。MI325X 已在 fixed_seq_len/qwen3.5_fp8_mi325x_mtp.sh 中使用完全相同的 MTP 配置；本 PR 新增的是该 SKU 上的智能体编码场景。

合成强制 AL：吞吐量测试导出 golden_al_distribution/README.md 规定的三个 SGLang 环境变量：SGLANG_SIMULATE_ACC_LEN=3.39（qwen3.5_mtp.yaml，thinking_on，K=3）、SGLANG_SIMULATE_ACC_METHOD=match-expected、SGLANG_SIMULATE_ACC_TOKEN_MODE=real-draft-token。EVAL_ONLY 不启用模拟接受、保留真实验证。

镜像 lmsysorg/sglang-rocm:v0.5.16-rocm720-mi30x-20260730（tag 已在 Docker Hub 验证）。SGLANG_SIMULATE_ACC_TOKEN_MODE 自 v0.5.16 起才被读取；固定序列长度的 MI325X 条目使用 v0.5.12-rocm720-mi30x，会静默忽略该约定。

回放路径沿用 MI355X 的 Qwen3.5 AgentX 配方而非固定序列长度的 gfx942 配方：--attention-backend triton（固定序列长度的 ROCm 配方用 aiter，但智能体语料是在 triton 上验证的）；HiCache 固定走 page_size=1 的 direct/layer_first 拷贝路径 —— Qwen3.5 的混合 Mamba 路径在 SGLang 上使用 no_buffer 调度器，而 ROCm 上 kernel/page_first 传输路径会在首次 prefill 时崩溃；并加上 --skip-server-warmup，由 aiperf 负责基准流量。--mem-fraction-static 取 0.75，与 fixed_seq_len/qwen3.5_fp8_mi325x.sh 在这块 256 GB SKU 上的设置一致。

搜索空间覆盖TP8 GPU 常驻与 HiCache 两条分支，并采用 AgentX MTP 并发网格：GPU 常驻 [1, 4, 8, 12, 16]，KV 卸载 [4, 8, 12, 16]。相邻并发间隔至少为 2，且一律止于并发 16 —— 在智能体语料上，逐 1 采样无法区分出超过运行间噪声的差异；而并发超过 16 后，这些 SKU 已进入非 MTP 扫描早已刻画过的显存瓶颈后抖动区间。共生成 9 个作业。

验证：bash -n 通过；两个 YAML 均可解析；generate_sweep_configs.py test-config 生成 9 条 cluster:mi325x-amds 上的 agentic-coding 配置。这是该 SKU 上的首次接入，建议评审确认并发区间与 triton/HiCache 组合后再启动扫描。

🤖 Generated with Claude Code

---

Revision

Concurrency grid. Steps of at least 2 and a hard stop at conc 16, replacing the sampling carried over from the non-MTP agentic shapes.

中文：并发网格调整为相邻间隔至少 2、一律止于并发 16，不再沿用非 MTP 智能体配方的采样方式。

Revision 3 — image swapped to the sglang-rocm build (Python 3.11)

The first job to actually start died ~30 s in, before serving:

uv pip install ... -e /workspace/utils/aiperf
Using Python 3.10.12 environment at: .../venv
x No solution found: the current Python version (3.10.12) does not satisfy Python>=3.11,<3.14
  and aiperf==0.12.0 depends on Python>=3.11,<3.14

lmsysorg/sglang:v0.5.16-rocm720-mi30x ships Python 3.10.12 in /opt/venv, and install_agentic_deps builds the AgentX venv from $(command -v python3), so uv can never resolve AIPerf. My earlier image bump verified the tag existed but not that it could run AgentX — that was the miss.

Now on lmsysorg/sglang-rocm:v0.5.16-rocm720-mi30x-20260730: same sglang 0.5.16 (so SGLANG_SIMULATE_ACC_TOKEN_MODE is still honored), same gfx942 arch, but the dated sglang-rocm family ships CPython 3.11.15 — confirmed from a green MI355X AgentX job running lmsysorg/sglang-rocm:v0.5.15.post1-rocm720-mi35x-20260719, which is the same build pipeline. Tag verified present on Docker Hub.

中文：首个真正启动的作业在约 30 秒后、尚未提供服务前即失败 —— lmsysorg/sglang:v0.5.16-rocm720-mi30x 的 /opt/venv 为 Python 3.10.12，而 install_agentic_deps 依据 $(command -v python3) 创建 AgentX 虚拟环境，uv 无法解析要求 Python>=3.11 的 AIPerf。此前的镜像升级只验证了 tag 存在，未验证其能运行 AgentX。现改用 lmsysorg/sglang-rocm:v0.5.16-rocm720-mi30x-20260730：sglang 仍为 0.5.16、架构仍为 gfx942，且该 dated 系列自带 CPython 3.11.15 —— 已由运行 lmsysorg/sglang-rocm:v0.5.15.post1-rocm720-mi35x-20260719 的成功 MI355X AgentX 作业确认，二者出自同一构建流程。Docker Hub 上已验证该 tag 存在。

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[functionstackx]

Golden-curve provenance note for reviewers. The pinned AL 3.39 comes from golden_al_distribution/qwen3.5_mtp.yaml, whose model key is qwen3.5-397b-a17b-nvfp4 — the curve was collected on the NVFP4 checkpoint on B300, while this recipe serves the FP8 checkpoint. There is no qwen3.5 FP8 curve committed in the repo, and the AgentX policy in golden_al_distribution/README.md keys the golden curve on model / thinking mode / draft length rather than on serving precision, so 3.39 is the only committed target available. The existing GB300 Qwen3.5 AgentX srt-slurm recipes pin the same 3.39, but they are FP4 — these are the first FP8 recipes to use that curve.

If the maintainers consider the curve precision-sensitive, the fix is a new SPEED-Bench collection on the FP8 checkpoint via speedbench-al.yml, not a hand-adjusted value here.

中文：本配方固定的 AL 3.39 取自 golden_al_distribution/qwen3.5_mtp.yaml，其模型键为 qwen3.5-397b-a17b-nvfp4 —— 该曲线是在 B300 上基于 NVFP4 权重测得的，而本配方服务的是 FP8 权重。仓库中没有提交 Qwen3.5 的 FP8 曲线，且 golden_al_distribution/README.md 的 AgentX 规则按「模型 / thinking 模式 / 草稿长度」而非推理精度来索引黄金曲线，因此 3.39 是目前唯一可用的既定目标值。现有 GB300 Qwen3.5 AgentX srt-slurm 配方同样固定为 3.39，但它们是 FP4；本批是首次将该曲线用于 FP8 配方。

若维护者认为该曲线对精度敏感，正确做法是通过 speedbench-al.yml 在 FP8 权重上重新采集一条 SPEED-Bench 曲线，而不是在此处手工调整数值。

[claude]

LGTM — the only finding is a documentation-only comment/changelog inaccuracy (nonexistent MI300X sibling reference), which doesn't affect the sweep configuration itself.

Extended reasoning...

Overview

This PR adds the first Qwen3.5 AgentX (agentic-coding) recipe on MI325X: one new benchmark script (qwen3.5_fp8_mi325x_mtp.sh), one amd-master.yaml entry, and one perf-changelog.yaml entry. The script closely mirrors two existing, already-validated recipes: the MI325X fixed-seq-len MTP entry (for the EAGLE speculative-decoding shape) and the MI355X Qwen3.5 AgentX/HiCache entry (for the triton attention backend and the page_size=1 direct/layer_first HiCache workaround for Qwen3.5's hybrid Mamba path on ROCm). No shared library code or launcher logic is modified — the change is additive and isolated to this one model/SKU/scenario combination.

Security risks

None. This is a benchmark configuration and shell script for internal sweep infrastructure — no auth, crypto, secrets, or externally-facing surface is touched. The image tag is pinned to a specific, stated-as-verified Docker Hub tag rather than a floating tag.

Level of scrutiny

Moderate. This is a first-on-SKU bring-up recipe, which the PR description itself flags for reviewer attention regarding the concurrency bands and the triton/HiCache combination. However, the actual technical choices (spec-decoding shape, HiCache page_size=1 workaround, mem-fraction-static) are all copied from already-proven sibling recipes rather than newly invented, and the script follows the same structural pattern (env var checks, HiCache sizing guards, server launch) used throughout this file. The one issue found — a documentation/comment inaccuracy about which sibling recipe the concurrency bands are compared against — doesn't affect what the sweep actually runs; it only affects a future reader's ability to verify the stated rationale.

Other factors

Validation steps (bash -n, YAML parsing, generate_sweep_configs.py test-config producing the expected 11 jobs) are described in the PR and are consistent with the config's structure. There are no outstanding review threads to address, and this is the first review pass on this PR.

[claude]

🟡 The sizing rationale ("one band wider than the MI300X sibling") repeated in the script header, the amd-master.yaml comment, and the perf-changelog entry references a Qwen3.5 agentic/AgentX recipe on MI300X that doesn't exist — only the fixed-seq-len qwen3.5-bf16-mi300x-sglang and qwen3.5-fp8-mi300x-sglang are present. If the MI355X hicache sibling (qwen3.5-fp8-mi355x-sglang-agentic-hicache) was actually meant, that recipe sweeps a wider band (GPU-resident 1-32, HiCache 16-64) than this MI325X entry (1-24 / 16-48) — the opposite of what's claimed. This is a documentation-only issue and doesn't affect the sweep configuration itself.

Extended reasoning...

The bug: All three places that justify this PR's concurrency search space — the script header comment in benchmarks/single_node/agentic/qwen3.5_fp8_mi325x_mtp.sh, the comment directly above the new qwen3.5-fp8-mi325x-sglang-agentic-mtp entry in configs/amd-master.yaml (lines 1498-1520), and the new entry in perf-changelog.yaml — state that the chosen bands (GPU-resident conc 1-24, HiCache conc 16-48) are "one band wider than the MI300X sibling" / "Conc bands run past the MI300X entry's." This implies there is an existing Qwen3.5 agentic-coding recipe on MI300X that was used as the sizing baseline.

Why the baseline doesn't exist: Grepping configs/amd-master.yaml for MI300X Qwen3.5 recipes turns up exactly two: qwen3.5-bf16-mi300x-sglang and qwen3.5-fp8-mi300x-sglang. Both use the fixed-seq-len scenario with conc-start: 4, conc-end: 64 — they are not agentic-coding recipes at all, and neither has a conc-list/band structure that could be described as narrower or wider in the sense this PR claims. There is no MI300X entry anywhere in the file with an agentic-coding scenario for Qwen3.5. The only Qwen3.5 agentic recipes in the whole config are on MI355X (qwen3.5-fp8-mi355x-sglang-agentic, qwen3.5-fp8-mi355x-sglang-agentic-hicache) plus this new MI325X entry and several NVIDIA (B200/B300/H100/GB300) recipes.

Step-by-step proof:

1. Search configs/amd-master.yaml for qwen3.5 entries with runner: mi300x or runner: cluster:mi300x* → only qwen3.5-bf16-mi300x-sglang (fixed-seq-len, conc 4-64) and qwen3.5-fp8-mi300x-sglang (fixed-seq-len, conc 4-64).
2. Search for any agentic-coding scenario paired with a Qwen3.5 model-prefix and an MI300X runner → zero matches.
3. Therefore "the MI300X sibling" that this recipe's bands are supposedly "one band wider than" cannot be identified — there's nothing on MI300X with a comparable band structure to compare against.
4. As a fallback, check whether the author meant the MI355X hicache sibling instead (same agentic-coding scenario shape, same kv-offloading arms): qwen3.5-fp8-mi355x-sglang-agentic-hicache sweeps GPU-resident conc-list: [1, 2, 4, 8, 16, 32] (max 32) and HiCache conc-list: [16, 32, 48, 64] (max 64).
5. Compare to this PR's MI325X entry: GPU-resident conc-list: [1, 2, 4, 8, 12, 16, 24] (max 24) and HiCache conc-list: [16, 24, 32, 48] (max 48).
6. 24 < 32 and 48 < 64 — the MI325X entry's bands are narrower than the MI355X sibling at the top end, not wider, directly contradicting the stated rationale even under this alternate reading.

Why nothing else catches this: The comment/rationale text isn't validated against the config file's contents by any tooling — bash -n, YAML parsing, and generate_sweep_configs.py test-config (all cited in the PR's own validation section) only check that the sweep configs are syntactically well-formed and generate the expected 11 jobs; none of them cross-check prose claims about sibling recipes. So a comment that references a nonexistent baseline sails through review checks cleanly, and a future engineer reading this recipe to understand why these specific bounds (24/48) were chosen has no way to verify or reproduce the reasoning — the actual justification ("MI325X has more per-GPU memory than MI300X") is directionally reasonable, but the specific numeric comparison it's anchored to doesn't exist.

Impact and fix: This is purely a documentation/comment defect — the conc-list values themselves are valid and the 11-job sweep will run and parse correctly regardless of the misleading comment. The fix is simply to correct the rationale text in all three locations (script header, yaml comment, changelog) to either drop the sibling comparison entirely and justify the bands purely from MI325X's own memory budget, or point to a real, currently-narrower baseline if one exists elsewhere in the repo.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30516207832
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30516207832

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30517386186
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30517386186

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30578079076
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30578079076