[https://nvbugs/6448152][perf] diagnose 2x admission without PP consensus#16517
[https://nvbugs/6448152][perf] diagnose 2x admission without PP consensus#16517chienchunhung wants to merge 4 commits into
Conversation
Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>
Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>
Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>
Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>
|
/bot run --disable-fail-fast --stage-list "GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-1" |
|
PR_Github #59815 [ run ] triggered by Bot. Commit: |
|
PR_Github #59815 [ run ] completed with state
|
|
/bot run --disable-fail-fast --stage-list "GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-1" |
|
PR_Github #59838 [ run ] triggered by Bot. Commit: |
|
PR_Github #59838 [ run ] completed with state
|
|
/bot run --disable-fail-fast --stage-list "GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-1" |
|
PR_Github #59879 [ run ] triggered by Bot. Commit: |
|
PR_Github #59879 [ run ] completed with state
|
|
/bot run --disable-fail-fast --stage-list "GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-1" |
|
PR_Github #60016 [ run ] triggered by Bot. Commit: |
|
PR_Github #60016 [ run ] completed with state
|
2 participants
Caution
DIAGNOSTIC ONLY — DO NOT MERGE.
Question
Does the conservative generation-side KV-transfer admission budget cause the GB300 DeepSeek-R1 throughput regression, and does PP context-transfer consensus become material after that budget is relaxed?
This is the 2× admission / PP consensus disabled arm. The matched consensus-enabled arm is #16518.
Controlled change
Start from the exact tested [https://nvbugs/6448152][perf] diagnose PP consensus traffic #16487 head (
1522ea7230), which produced 798.58 output tokens/s with the one-window budget and runtime PP context consensus traffic disabled.Keep
max_tokens_in_buffer: 131104; physical C++/NIXL transfer-buffer allocation is unchanged.Apply a diagnostic-only multiplier to PyExecutor's logical admission budget:
2 * ceil(max_tokens_in_buffer / tokens_per_block)Keep
TRTLLM_DIAGNOSTIC_DISABLE_PP_CONTEXT_CONSENSUS_TRAFFIC=1enabled.Leave the diagnostic multiplier default at
1; only the exact GB300 perf-sanity YAML sets it to2.For this FlashMLA workload, the expected base budget is 2,049 blocks. A 131,072-token request costs 2,048 blocks, so the effective 4,098-block budget admits two full transfers (4,096 blocks) without resizing the transport buffer.
The run must contain both markers:
The second marker is one-shot and proves that the expanded budget was exercised, rather than merely configured.
Experiment matrix
The two 2× arms share the same code and differ only by the no-consensus environment flag. If their results are unexpectedly close to a threshold, rerun a same-harness 1× control to remove the negligible fast-return diagnostic method-call delta from the historical controls.
flowchart LR A["#16487 tested anchor<br/>1× / consensus off"] --> H["common logical 2× admission harness"] H --> OFF["#16517<br/>2× / consensus off"] H --> ON["#16518<br/>2× / consensus on"]Interpretation
Validation
transformersand the compiled TRT-LLM runtime; CI owns runtime validation.Target CI
Run only:
GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-1Compare output-token throughput with 798.58 (#16487) and 1562.83 baseline.