launcher: bump TRT-LLM to 1.3.0rc20, pin vLLM to v0.22.0, fix max_seq…#1982
Open
noeyy-mino wants to merge 2 commits into
Open
launcher: bump TRT-LLM to 1.3.0rc20, pin vLLM to v0.22.0, fix max_seq…#1982noeyy-mino wants to merge 2 commits into
noeyy-mino wants to merge 2 commits into
Conversation
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughLauncher defaults and example Slurm workflows now use updated TensorRT-LLM and vLLM container images. Qwen3.5-4B throughput benchmarks explicitly set a maximum sequence length of ChangesLauncher configuration
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 6✅ Passed checks (6 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
⚔️ Resolve merge conflicts
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash.yaml`:
- Line 93: Pin the vLLM smoke-test container from nightly to v0.22.0 in both
tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash.yaml:93-93 and
tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash_multi_node.yaml:96-96
by updating each container reference to the v0.22.0 image, using the multi-arch
variant if that is the established image elsewhere in the PR.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 8a333021-0ee0-4c0b-bc03-754659162969
📒 Files selected for processing (21)
tools/launcher/examples/Qwen/Qwen3-30B-A3B/hf_streaming_dflash_multi_node.yamltools/launcher/examples/Qwen/Qwen3-30B-A3B/hf_streaming_eagle3_multi_node.yamltools/launcher/examples/Qwen/Qwen3-8B/hf_eagle3_dryrun.yamltools/launcher/examples/Qwen/Qwen3-8B/hf_online_eagle3.yamltools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash.yamltools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash_multi_node.yamltools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_eagle3.yamltools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_eagle3_multi_node.yamltools/launcher/examples/Qwen/Qwen3.5-4B/specdec_bench.yamltools/launcher/examples/Qwen/Qwen3.5-4B/specdec_bench_mtp_vllm.yamltools/launcher/examples/moonshotai/Kimi-K2.5/hf_dflash_dryrun.yamltools/launcher/examples/moonshotai/Kimi-K2.5/hf_eagle3_dryrun.yamltools/launcher/examples/moonshotai/Kimi-K2.5/hf_offline_dflash.yamltools/launcher/examples/moonshotai/Kimi-K2.5/hf_streaming_dflash.yamltools/launcher/examples/moonshotai/Kimi-K2.5/hf_streaming_dflash_multi_node.yamltools/launcher/examples/moonshotai/Kimi-K2.5/hf_streaming_eagle3.yamltools/launcher/examples/moonshotai/Kimi-K2.5/hf_streaming_eagle3_multi_node.yamltools/launcher/examples/moonshotai/Kimi-K2.5/specdec_bench.yamltools/launcher/examples/openai/gpt-oss-20b/hf_streaming_dflash_multi_node.yamltools/launcher/examples/openai/gpt-oss-20b/hf_streaming_eagle3_multi_node.yamltools/launcher/slurm_config.py
| slurm_config: | ||
| _factory_: "slurm_factory" | ||
| container: "vllm/vllm-openai:nightly" | ||
| container: vllm/vllm-openai:nightly |
Contributor
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
vLLM smoke-test containers still float on nightly, not pinned to v0.22.0.
The PR objective states vLLM is being pinned to v0.22.0, but these two workflows still reference the unpinned vllm/vllm-openai:nightly tag for their vLLM smoke-test task, making runs non-reproducible.
tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash.yaml#L93-L93: pincontainertovllm/vllm-openai:v0.22.0(or the multi-arch v0.22.0 image used elsewhere in this PR).tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash_multi_node.yaml#L96-L96: same fix — pin tovllm/vllm-openai:v0.22.0.
📍 Affects 2 files
tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash.yaml#L93-L93(this comment)tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash_multi_node.yaml#L96-L96
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash.yaml` at line 93,
Pin the vLLM smoke-test container from nightly to v0.22.0 in both
tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash.yaml:93-93 and
tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash_multi_node.yaml:96-96
by updating each container reference to the v0.22.0 image, using the multi-arch
variant if that is the established image elsewhere in the PR.
Contributor
Author
There was a problem hiding this comment.
Hi @ChenhanYu @h-guo18， For the vLLM container, can we use the latest tag—i.e., vllm/vllm-openai:latest—so that it always points to the newest vLLM release version? This way, we wouldn’t need to frequently update the YAML file.
noeyy-mino
force-pushed
the
noeyy/bump-containers-fix-max-seq-len
branch
from
9c8b5af to
1748dbf
Compare
…_len for Qwen3.5-4B Signed-off-by: noeyy-mino <174223378+noeyy-mino@users.noreply.github.com>
Signed-off-by: noeyy-mino <174223378+noeyy-mino@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1982 +/- ##
==========================================
+ Coverage 76.59% 76.62% +0.03%
==========================================
Files 525 525
Lines 59253 59253
==========================================
+ Hits 45385 45404 +19
+ Misses 13868 13849 -19
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bump TRT-LLM to 1.3.0rc20, pin vLLM to v0.22.0, fix max_seq_len for Qwen3.5-4B
What does this PR do?
Type of change: Bug fix, new feature
Qwen3-8B, Qwen3-30B-A3B, Kimi-K2.5, and gpt-oss-20b launcher configs.
the multi-arch v0.22.0 tag, which resolves to amd64/arm64 automatically.
65536 to accommodate outlier prompts (~46.6K tokens) that caused
VLLMValidationError and aborted the entire benchmark run.
runtime_params_throughput_32k.yaml; use --max_seq_len 65536 instead
Usage
Testing
N/A
CONTRIBUTING.md: N/AAdditional Information
N/A
Summary by CodeRabbit
1.3.0rc20image.0.22.0image (removing prior architecture-specific variants).