Add Qwen3.5 FP4 B300 AgentX MTP / 添加 Qwen3.5 FP4 B300 AgentX MTP - #2421
Add Qwen3.5 FP4 B300 AgentX MTP / 添加 Qwen3.5 FP4 B300 AgentX MTP#2421cquil11 wants to merge 37 commits into
Conversation
Add the initial SGLang NEXTN AgentX recipe with golden synthetic acceptance and 256k traces.\n\n中文：新增 Qwen3.5 FP4 B300 AgentX MTP 初始配置，使用 SGLang NEXTN、黄金合成接受长度和 256k 轨迹数据集。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Append the Qwen3.5 FP4 B300 AgentX MTP benchmark entry.\n\n中文：追加 Qwen3.5 FP4 B300 AgentX MTP 基准测试触发记录。
cquil11
changed the title
Add strict two-pool HiCache accounting, cache reporting, session-sticky DEP8 routing, AgentX request/graph limits, and measured-cache guidance for the B300 frontier.\n\n中文：为 B300 前沿调优加入严格的双池 HiCache 容量核算、缓存统计、会话粘性的 DEP8 路由、AgentX 请求与 CUDA Graph 上限，并记录实测缓存容量依据。
Add isolated TP2 c72 and TP4 c80 DRAM HiCache points so each cache-cliff recovery topology can be measured independently. 中文：添加独立的 TP2 c72 与 TP4 c80 DRAM HiCache 测试点，以便分别测量各拓扑在缓存工作集拐点后的恢复效果。
Account for SGLang target KV, Mamba, and native NEXTN draft host pools as H * 31/15 per rank, with page-alignment reserve, so projected use stays within runner-generated DRAM. 中文：按每个 rank 的 H * 31/15 计入 SGLang 目标 KV、Mamba 与原生 NEXTN 草稿主机缓存，并预留页对齐空间，确保预计用量不超过 runner 生成的 DRAM 上限。
中文：将最新 main 合并到 B300 提交分支，并将本 PR 的性能变更记录重新追加到文件末尾。
Add TP4 concurrency points around the measured HiCache crossover and extend the DRAM-offload curve beyond it.\n\n中文：在实测 HiCache 切换区间细化 B300 Qwen TP4 并发点，并将 DRAM 卸载曲线扩展到拐点之后。
Remove the measured no-offload thrashing tail and add dense TP2 points across the expected HiCache crossover.\n\n中文：移除实测发生抖动的无卸载尾部配置，并在预期 HiCache 切换区间细化 B300 TP2 并发点。
Drop the attention-DP arm after live AgentX profiling showed fragmented prefix caches and throughput below the TP frontier. Remove the now-unused router path.\n\n中文：AgentX 实测表明注意力 DP 会导致前缀缓存碎片化且吞吐低于 TP 前沿，因此移除该配置及不再使用的路由路径。
Probe three post-c72 offload points to locate saturation of the measured TP2 efficiency frontier.\n\n中文：在 c72 之后增加三个卸载并发点，用于定位实测 B300 TP2 效率前沿的饱和位置。
Remove the no-offload c96 point after it was dominated by HiCache c80 in both throughput and latency, and refresh the measured crossover note.\n\n中文：无卸载 c96 在吞吐与延迟上均被 HiCache c80 支配，因此移除该点并更新实测切换区间说明。
Remove offload points that exceed the measured host-cache validity boundary and densify the final safe interval below c80.\n\n中文：移除超过实测主机缓存有效边界的卸载配置，并在 c80 以下的安全区间细化最终并发点。
Remove no-offload c64 after it stalled under the long-context working set; retain the matched HiCache points beyond c56.\n\n中文：无卸载 c64 在长上下文工作集下停滞，因此移除该点，并保留 c56 之后对应的 HiCache 配置。
中文：在缓存工作集拐点测试 B300 TP4 HiCache。
中文：裁剪 B300 TP2 无卸载配置的缓存拐点尾部。
中文：优化 B300 缓存悬崖附近的并发点，并使用 SGLang 默认单 tokenizer 启动路径以避免 HiCache 初始化期间的多 tokenizer 共享内存竞态。
中文：在 B300 HiCache 缓存悬崖附近增加 c84、c92 和 c94 测试点，以更细粒度覆盖有效的帕累托前沿。
中文：在 B300 TP2 的 c32 至 c56 区间增加 HiCache 重叠测试点，以覆盖一小时 AgentX 预热下可能提前出现的缓存悬崖。
Use six tokenizer workers for TP4 long-context replay while retaining SGLang default single-worker startup for TP2 HiCache. 中文：TP4 长上下文回放使用 6 个 tokenizer worker；TP2 HiCache 保持 SGLang 默认单 worker 启动路径，避免共享内存初始化竞态。
Add HiCache concurrency 34 and 38 around the measured c36-to-c40 transition. 中文：在实测 c36 到 c40 的转折区间补充 HiCache 并发点 34 和 38。
Densify the TP4 no-offload cliff and remove TP4 HiCache points that are dominated by the TP4 no-offload and TP2 HiCache arms. Keep TP2 HiCache dense through the measured host-KV saturation region.\n\n中文：细化 B300 缓存帕累托前沿；加密 TP4 无卸载缓存临界区间，移除被 TP4 无卸载与 TP2 HiCache 方案支配的 TP4 HiCache 测点，并保留覆盖主机 KV 饱和区间的密集 TP2 HiCache 测点。
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30565108920 |
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30569690916 |
Update AIPerf after removing runtime trace idle enforcement. Signed-off-by: Cam Quilici <cjquilici@gmail.com>
cquil11
added
agentx-fast
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30571065906 |
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30577529606 |
|
/stage-results 30577529606 |
|
@cquil11 staged run 30577529606: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-07-30~r30577529606 This run remains available across future @cquil11 已将运行 30577529606 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-07-30~r30577529606 后续的 |
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
cquil11
removed
the
agentx-fast
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30584325544 |
|
/stage-results 30590079878 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30590079878 |
|
@cquil11 staged run 30590079878: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-07-30~r30590079878 This run remains available across future @cquil11 已将运行 30590079878 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-07-30~r30590079878 后续的 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30597337939 |
|
/stage-results 30597337939 |
|
@cquil11 staged run 30597337939: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-07-31~r30597337939 This run remains available across future @cquil11 已将运行 30597337939 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-07-31~r30597337939 后续的 |
中文：将 AIPerf 子模块固定到已验证的 AgentX 交接、请求顺序与父子会合计时修复版本。
cquil11
changed the title
Pin AIPerf to the validated revision that preserves join dependencies while enforcing the whole-system idle cap across replay waits. 中文：将 AIPerf 固定到已验证版本，在保留子代理汇合依赖的同时，对所有回放等待正确执行全局空闲上限。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30610065067 |
1 participant
Summary
Add Qwen3.5-397B-A17B NVFP4 AgentX coverage on B300 with SGLang native NEXTN MTP, golden synthetic acceptance, prefix caching, the 256k trace dataset, and a 300-second per-trace idle-gap cap. Pin the validated AIPerf handoff, request-order, and parent/child join-timing fixes.
中文说明
新增 Qwen3.5-397B-A17B NVFP4 在 B300 上的 AgentX 基准测试，使用 SGLang 原生 NEXTN MTP、经验证的合成接受率、前缀缓存、256k 轨迹数据集，以及每条轨迹 300 秒的空闲间隔上限。AIPerf 子模块固定到已验证的交接、请求顺序与父子会合计时修复版本。