[None][test] Add DeepSeek-V4-Pro perf sanity cases on GB300#16540
[None][test] Add DeepSeek-V4-Pro perf sanity cases on GB300#16540chenfeiz0326 wants to merge 4 commits into
Conversation
📝 WalkthroughWalkthroughAdds role-specific context/generation environment handling, a DeepSeek-V4-Pro model mapping, GB300 disaggregated PerfSanity workloads, post-merge scheduling entries, and fixed MoE load-balancer assignment configurations. ChangesGB300 PerfSanity
Estimated code review effort: 3 (Moderate) | ~30 minutes Sequence Diagram(s)sequenceDiagram
participant DisaggYAML
participant PerfSanityTestConfig
participant ServerConfig
participant submit_py
DisaggYAML->>PerfSanityTestConfig: provide shared and role-specific environment values
PerfSanityTestConfig->>ServerConfig: assign context and generation environments
submit_py->>submit_py: compose role-specific launch prefixes
submit_py->>ServerConfig: generate disaggregated worker launch environments
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@jenkins/L0_Test.groovy`:
- Around line 4963-4966: Update the stage-name performance checks in both the
x86 and SBSA Slurm job closures to recognize names containing “-PerfSanity-” as
performance tests alongside the existing “-Perf-” condition. Ensure the
resulting perfMode passed to runLLMTestlistOnSlurm is true for these stages so
the --perf workflow is preserved.
In `@jenkins/scripts/perf/submit.py`:
- Around line 538-541: Update the base_prefix construction to remove the
unnecessary f-string prefixes while preserving the generated shell variable
syntax. Convert the doubled braces around SLURM_LOCALID to literal braces so the
resulting command still contains ${SLURM_LOCALID} and satisfies Ruff F541.
In
`@tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx12_node1_gpu4_gen1_node2_gpu8.yml`:
- Around line 1-2: Add the repository’s standard NVIDIA copyright header at the
beginning of the YAML file, before the version declaration. Preserve the
existing version and configuration key unchanged.
In
`@tests/scripts/perf-sanity/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con8_ctx1_dep4_gen4_tep8_eplb0_mtp3_ccb-NIXL.yaml`:
- Around line 1-2: Add the repository-standard NVIDIA copyright header at the
beginning of each new YAML file:
tests/scripts/perf-sanity/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con8_ctx1_dep4_gen4_tep8_eplb0_mtp3_ccb-NIXL.yaml
(lines 1-2),
tests/scripts/perf-sanity/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con4301_ctx12_dep4_gen1_dep8_eplb384_mtp1_ccb-NIXL.yaml
(lines 1-2),
tests/scripts/perf-sanity/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con666_ctx6_dep4_gen1_dep16_eplb384_mtp3_ccb-NIXL.yaml
(lines 1-2),
tests/scripts/perf-sanity/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yaml
(lines 1-2),
tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen4_node2_gpu8.yml
(lines 1-2),
tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx6_node1_gpu4_gen1_node4_gpu16.yml
(lines 1-2), and
tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx3_node1_gpu4_gen1_node8_gpu32.yml
(lines 1-2), while preserving each file’s existing YAML content.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 8e14155b-d131-457e-95ad-9d4ac5ac36da
📒 Files selected for processing (23)
jenkins/L0_Test.groovyjenkins/scripts/perf/local/submit.pyjenkins/scripts/perf/submit.pytests/integration/defs/perf/_model_paths.pytests/integration/defs/perf/test_perf_sanity.pytests/integration/test_lists/test-db/l0_gb300_multi_gpus_perf_sanity.ymltests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx12_node1_gpu4_gen1_node2_gpu8.ymltests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen4_node2_gpu8.ymltests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx3_node1_gpu4_gen1_node8_gpu32.ymltests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx6_node1_gpu4_gen1_node4_gpu16.ymltests/scripts/perf-sanity/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_ctx_ep4_384.yamltests/scripts/perf-sanity/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep16_slots384.yamltests/scripts/perf-sanity/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep16_slots416.yamltests/scripts/perf-sanity/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots384.yamltests/scripts/perf-sanity/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots416.yamltests/scripts/perf-sanity/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep4_slots384.yamltests/scripts/perf-sanity/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep4_slots416.yamltests/scripts/perf-sanity/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep8_slots384.yamltests/scripts/perf-sanity/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep8_slots416.yamltests/scripts/perf-sanity/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yamltests/scripts/perf-sanity/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con4301_ctx12_dep4_gen1_dep8_eplb384_mtp1_ccb-NIXL.yamltests/scripts/perf-sanity/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con666_ctx6_dep4_gen1_dep16_eplb384_mtp3_ccb-NIXL.yamltests/scripts/perf-sanity/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con8_ctx1_dep4_gen4_tep8_eplb0_mtp3_ccb-NIXL.yaml
BowenFu
left a comment
There was a problem hiding this comment.
LGTM — additive perf-sanity test/CI (DeepSeek-V4-Pro on GB300); new ctx/gen worker_env_var yaml keys are backward-compatible; existing yamls unaffected.
|
/bot run --disable-fail-fast --stage-list "GB300-36_GPUs-9_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN4-NODE2-GPU8-Post-Merge-1,GB300-40_GPUs-10_Nodes-PyTorch-Disagg-PerfSanity-CTX6-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge-1,GB300-44_GPUs-11_Nodes-PyTorch-Disagg-PerfSanity-CTX3-NODE1-GPU4-GEN1-NODE8-GPU32-Post-Merge-1,GB300-56_GPUs-14_Nodes-PyTorch-Disagg-PerfSanity-CTX12-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-1" |
Adds four end-to-end disaggregated perf-sanity cases for DeepSeek-V4-Pro
(FP4, 1.6T MoE) on GB300, covering the concurrency sweep (single-user
latency at con=8, mid throughput at con=180 and con=666, max throughput
at con=4301).
New per-role env-var support
- environment.ctx_worker_env_var and environment.gen_worker_env_var yaml
keys (both optional). Merged with the shared environment.worker_env_var
at ServerConfig construction time in test_perf_sanity.py and in both
submit scripts (jenkins/scripts/perf/{local/,}submit.py). Existing yamls
with only worker_env_var are unaffected.
- Used by the new DSv4 configs to apply
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True on the ctx worker only
(long-prompt path where expandable_segments avoids CUDA fragmentation).
New DSv4-Pro configs (all 8k1k, GB300, NIXL, dep4 ctx / MegaMoE gen)
- gb300_deepseek-v4-pro-fp4_8k1k_con8_ctx1_dep4_gen4_tep8_eplb0_mtp3 (1 ctx x 4GPU + 4 gen x 8GPU = 9 nodes / 36 GPUs)
- gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3 (3 ctx + 1 gen x 32GPU = 11 nodes / 44 GPUs)
- gb300_deepseek-v4-pro-fp4_8k1k_con666_ctx6_dep4_gen1_dep16_eplb384_mtp3 (6 ctx + 1 gen x 16GPU = 10 nodes / 40 GPUs)
- gb300_deepseek-v4-pro-fp4_8k1k_con4301_ctx12_dep4_gen1_dep8_eplb384_mtp1 (12 ctx + 1 gen x 8GPU = 14 nodes / 56 GPUs)
Each config is registered in three test modes (aggr_upload-ctx_only-*,
disagg_upload-gen_only-*, disagg_upload-e2e-*) via the aggregated test-db
(l0_gb300_multi_gpus_perf_sanity.yml, split bumped 3 -> 4) and four new
multi-node test-db yamls (one per shape).
Companion pieces
- tests/integration/defs/perf/_model_paths.py: add
deepseek_v4_pro_fp4 -> DeepSeek-V4-Pro
- test_perf_sanity.py generate_extra_llm_api_config: resolve
moe_config.load_balancer relative paths against get_llm_root(), so
configs can reference the in-repo eplb yamls.
- tests/scripts/perf-sanity/disaggregated/deepseek-v4-pro-eplb/: nine
offline moe-load-balancer yamls (ctx_ep4_384 + gen_ep{4,8,16,32}_slots{384,416})
cherry-picked from bench-trtllm-disagg so CI does not need external
filesystem access.
- jenkins/L0_Test.groovy: four new buildStageConfigs entries under the
GB300 disagg section (9, 10, 11, 14 nodes), plus the aggregated split
bump.
Signed-off-by: chenfeiz <chenfeiz@nvidia.com>
Signed-off-by: Chenfei Zhang <chenfeiz@nvidia.com>
…erverConfig
The new DeepSeek-V4-Pro configs added in the previous commit reference the
in-repo eplb YAMLs via a repo-relative path string (moe_config.load_balancer:
tests/scripts/perf-sanity/disaggregated/deepseek-v4-pro-eplb/...yaml). The
TRT-LLM engine accepts either an inline dict or a path string via
MoeLoadBalancerConfig, and generate_extra_llm_api_config already resolves
the relative path to an absolute one before writing the extra-llm-api-config
yaml. But ServerConfig.__init__ was still treating the field as always-dict
and crashed at pytest collection:
perf/test_perf_sanity.py:470: AttributeError:
'str' object has no attribute 'get'
Add an isinstance(str) branch that zeros out the inline load_balancer_num_slots
and load_balancer_layer_updates_per_iter — those metrics live inside the
referenced YAML and aren't scraped by perf-sanity's DB fields anyway.
Signed-off-by: chenfeiz <chenfeiz@nvidia.com>
Signed-off-by: Chenfei Zhang <chenfeiz@nvidia.com>
- Drop disagg_upload-gen_only-* entries from the four new GB300 test-db yamls; keep only the disagg_upload-e2e-* entry. gen_only variants are deferred to a follow-up PR so this PR ships the single mode the concurrency-sweep table calls for. - Bump each new buildStageConfigs testCount from 2 → 1 in L0_Test.groovy now that each test-db yaml holds exactly one active test. - Drop the extraneous f-prefix on the base_prefix string in jenkins/scripts/perf/submit.py's disagg env-var block (Ruff F541). Signed-off-by: chenfeiz <chenfeiz@nvidia.com> Signed-off-by: Chenfei Zhang <chenfeiz@nvidia.com>
Fold the two-line string concat into a single line so pre-commit's line-length / auto-fold formatter passes. Signed-off-by: chenfeiz <chenfeiz@nvidia.com> Signed-off-by: Chenfei Zhang <chenfeiz@nvidia.com>
a529cf9 to
f846061
Compare
|
/bot run --disable-fail-fast --stage-list "GB300-36_GPUs-9_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN4-NODE2-GPU8-Post-Merge-1,GB300-40_GPUs-10_Nodes-PyTorch-Disagg-PerfSanity-CTX6-NODE1-GPU4-GEN1-NODE4-GPU16-Post-Merge-1,GB300-44_GPUs-11_Nodes-PyTorch-Disagg-PerfSanity-CTX3-NODE1-GPU4-GEN1-NODE8-GPU32-Post-Merge-1,GB300-56_GPUs-14_Nodes-PyTorch-Disagg-PerfSanity-CTX12-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-1" |
|
PR_Github #60001 [ run ] triggered by Bot. Commit: |
|
PR_Github #60001 [ run ] completed with state
|
Summary
Add four end-to-end DeepSeek-V4-Pro (FP4, 1.6T MoE) disaggregated perf-sanity cases on GB300, covering the concurrency sweep:
Each config runs in three test modes (aggr_upload-ctx_only-, disagg_upload-gen_only-, disagg_upload-e2e-*), so this PR ends up with 4 aggregated + 8 multi-node stages.
What's in this PR
New yaml schema keys (backward-compatible) — environment.ctx_worker_env_var and environment.gen_worker_env_var — merged with the shared environment.worker_env_var in test_perf_sanity.py and both submit scripts (jenkins/scripts/perf/{,local/}submit.py). Existing yamls with only worker_env_var are unaffected. Used by these configs to apply PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True on the ctx worker only (long-prompt path).
4 new disagg configs under tests/scripts/perf-sanity/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_*.yaml. Use `<dataset_file>` placeholder → random synthetic dataset at runtime (mirrors GLM-5 perf-sanity pattern).
Model registration — deepseek_v4_pro_fp4 → DeepSeek-V4-Pro in tests/integration/defs/perf/_model_paths.py.
In-repo MoE load-balancer yamls — tests/scripts/perf-sanity/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_{ctx_ep4_384, gen_ep{4,8,16,32}_slots{384,416}}.yaml (nine files, cherry-picked from ibc/bench-trtllm-disagg) so CI doesn't need external filesystem access. test_perf_sanity.py's generate_extra_llm_api_config now resolves relative moe_config.load_balancer paths against get_llm_root().
Test-db entries — 4 new l0_gb300_multi_nodes_perf_sanity_ctx{1,3,6,12}_node1_gpu4_gen*_node*_gpu*.yml (each holding one e2e + one gen_only), plus 4 new aggr_upload-ctx_only-* entries in l0_gb300_multi_gpus_perf_sanity.yml (split bumped 3 → 4).
Jenkins wiring — 4 new buildStageConfigs blocks in the GB300 disagg section of L0_Test.groovy (9, 10, 11, 14 nodes).
Test plan
Notes
Summary by CodeRabbit
New Features
Bug Fixes