Add B300 config: kimi-k2.5-fp4-vllm

[cquil11]

Warning

This PR is a reopen of #1056, which was reverted in #1099 due to an error with the first PR. The contents and description are otherwise identical to the original.

Summary

• Add kimik2.5-fp4-b300-vllm benchmark config and the corresponding benchmarks/single_node/kimik2.5_fp4_b300.sh launch script
• At the time of submission, the vLLM Kimi-K2.5 recipes page does not have a B300-specific recipe, so this reuses the existing Kimi-K2.5 FP4 (NVFP4) B200 vLLM recipe as-is until B300-specific tuning is available
• Image: vllm/vllm-openai:v0.17.0 (same as B200), runner: b300, same TP/EP/concurrency search-space as B200

Test plan

• CI config validation passes
• Run kimik2.5-fp4-b300-vllm single-node benchmark on a B300 node and confirm server starts, benchmark completes, and result file is produced

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://cookbook.sglang.io/
• https://docs.vllm.ai/projects/recipes/en/latest/

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sgl-cookbook

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[claude]

LGTM — straightforward B300 config addition following established patterns; the stale pr-link in the changelog entry is a minor nit already flagged inline.

Extended reasoning...

Overview

This PR adds a new benchmark configuration kimik2.5-fp4-b300-vllm to three files: .github/configs/nvidia-master.yaml (new config block), benchmarks/single_node/kimik2.5_fp4_b300.sh (new launch script), and perf-changelog.yaml (new changelog entry). It is a direct re-addition of the change from PR #1056, which was reverted by PR #1099.

Security risks

None. These are benchmark configuration and shell script changes with no auth, crypto, or permission-sensitive code paths. The shell script follows the same pattern as all other single-node benchmark scripts in the repo.

Level of scrutiny

Low. The config block mirrors the existing kimik2.5-fp4-b200-vllm entry with only the runner changed to b300 and the image bumped to v0.19.0-cu130. The launch script is a copy of the B200 script with the same vLLM serve invocation. Comparable B300 entries for MiniMax-M2.5 FP4/FP8 (PRs #1054, #1055) used the identical approach.

Other factors

The only bug found is a [Nit]: the perf-changelog.yaml entry carries pr-link: .../pull/1056 (the reverted PR) instead of pull/1100. This is a cosmetic tracking issue that doesn't affect CI, benchmarks, or functionality. The inline comment already captures this.

[claude]

🟡 The perf-changelog.yaml entry for kimik2.5-fp4-b300-vllm has pr-link pointing to PR #1056, which was explicitly reverted by PR #1099. Since this PR (#1100) is the one that will actually land the change, the pr-link should reference https://github.com/SemiAnalysisAI/InferenceX/pull/1100 instead.

Extended reasoning...

What the bug is and how it manifests

The new perf-changelog.yaml entry for kimik2.5-fp4-b300-vllm (added in this PR) sets pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1056. PR #1056 was the original submission of this B300 config, but it was subsequently reverted by PR #1099 due to an error. The PR description for #1100 explicitly states: "This PR is a reopen of #1056, which was reverted in #1099 due to an error with the first PR." This means PR #1056 exists on GitHub in a reverted/superseded state, while PR #1100 is the change that will actually merge the configuration into main.

The specific code path that triggers it

In the diff, the new changelog block reads:

pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1056

This was carried over from the original PR #1056 submission without being updated to reflect the current PR number (#1100).

Why existing code does not prevent it

The changelog is a manually maintained YAML file with no automated validation that cross-checks pr-link values against the actual PR being merged. There is no CI check to enforce that the link matches the current PR. The error is a straightforward copy-paste oversight when reopening the PR.

What the impact would be

When anyone later looks up the changelog entry for kimik2.5-fp4-b300-vllm to understand when and how it was introduced, they will follow the pr-link and land on PR #1056, which GitHub shows as reverted. This is misleading — it appears the feature was reverted rather than merged. The actual merging PR (#1100) would not be linked anywhere in the changelog, making historical tracking inaccurate.

How to fix it

Change the pr-link in the new changelog entry from pull/1056 to pull/1100.

Step-by-step proof

1. PR Add B300 config: kimi-k2.5-fp4-vllm #1056 was opened and merged, adding the kimik2.5-fp4-b300-vllm config.
2. PR Revert "Add B300 config: kimi-k2.5-fp4-vllm (#1056)" [skip-sweep] #1099 explicitly reverted PR Add B300 config: kimi-k2.5-fp4-vllm #1056 (commit: "Revert 'Add B300 config: kimi-k2.5-fp4-vllm (Add B300 config: kimi-k2.5-fp4-vllm #1056)'").
3. PR Add B300 config: kimi-k2.5-fp4-vllm #1100 (this PR) reopens the same change — its description confirms: "This PR is a reopen of Add B300 config: kimi-k2.5-fp4-vllm #1056, which was reverted in Revert "Add B300 config: kimi-k2.5-fp4-vllm (#1056)" [skip-sweep] #1099".
4. The new changelog entry in this PR's diff sets pr-link to /pull/1056 (the reverted PR) instead of /pull/1100 (this PR).
5. Comparable B300 entries in the same file correctly reference their own PR numbers: minimaxm2.5-fp8-b300-vllm -> Add B300 config: minimaxm2.5-fp8-vllm #1054, minimaxm2.5-fp4-b300-vllm -> Add B300 config: minimaxm2.5-fp4-vllm #1055, dsr1-fp8-b300-sglang-mtp -> Add B300 config: dsr1-fp8-sglang-mtp #1059.
6. The pr-link should therefore be updated to https://github.com/SemiAnalysisAI/InferenceX/pull/1100.

[xinli-sw]

Hi @cquil11 , I think we could do two things to get better perf

1. B300 could benefit from more parallel configs, we can run some more and report back
2. We should also include changes from https://github.com/SemiAnalysisAI/InferenceX/pull/1047/changes

Depending on how fast you want the data, we can either have #1407 as a fast follow of this PR or just wait until that's ready. Thanks!

[cquil11]

@xinli-sw No real rush, we can wait for y'all.

[functionstackx]

hi @xinli-sw it would be if https://recipes.vllm.ai/ had b300 specific configs so that ppl can know what flags to use specifically for b300

[xinli-sw]

yup for sure, I think mostly just having better parallel configs so that B300's higher mem size, will update shortly

[functionstackx]

@claude look at the fix in #1104 for b200, add it to this b300 PR too

[Klaud-Cold]

Claude finished @functionstackx's task in 2m 14s —— View job

---

Applying B200 fix from PR #1104 to this B300 PR

• Look at PR Bump max-model-len headroom from +200 to +256 (block-size alignment) #1104 diff to understand the B200 fix
• Identify what needs to change in this B300 PR
• Verify current state

Result

The fix from PR #1104 (bump max-model-len headroom from +200 to +256 for block-size alignment) is already included in this branch via commit 69a2584:

File	Status
benchmarks/benchmark_lib.sh (setup_eval_context)	+256
benchmarks/single_node/gptoss_fp4_b200.sh	+256
benchmarks/single_node/gptoss_fp4_h200.sh	+256
utils/matrix_logic/generate_sweep_configs.py (4 sites)	+256
utils/matrix_logic/test_generate_sweep_configs.py	+256

The B300 kimi script (kimik2.5_fp4_b300.sh) itself doesn't need a direct change because it receives MAX_MODEL_LEN from the environment variable set by generate_sweep_configs.py, which already uses the +256 headroom. No additional changes needed.
|