Skip to content

launcher: bump TRT-LLM to 1.3.0rc20, pin vLLM to v0.22.0, fix max_seq… #1982

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
noeyy-mino wants to merge 2 commits into
base: main
Choose a base branch
from
Open

launcher: bump TRT-LLM to 1.3.0rc20, pin vLLM to v0.22.0, fix max_seq… #1982

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
041c758
launcher: bump TRT-LLM to 1.3.0rc20, pin vLLM to v0.22.0, fix max_seq…
noeyy-mino Jul 16, 2026
1748dbf
upgrade TRT LLM container
noeyy-mino Jul 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion tools/launcher/examples/Qwen/Qwen3-30B-A3B/hf_streaming_dflash_multi_node.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 2: Streaming DFlash training — 2 serve replicas (TP=2) + 2 trainer nodes (2 GPU each).
# DFlash extracts 5 target layers (build_target_layer_ids(48,5)=[1,12,23,34,45], the
Expand Down
2 changes: 1 addition & 1 deletion tools/launcher/examples/Qwen/Qwen3-30B-A3B/hf_streaming_eagle3_multi_node.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 2: Streaming EAGLE3 training — 2 serve replicas (TP=2) + 2 trainer nodes (2 GPU each).
# Capture ids: default_eagle_aux_layer_ids(48)=[1,23,44] +1, plus final layer 48.
Expand Down
8 changes: 4 additions & 4 deletions tools/launcher/examples/Qwen/Qwen3-8B/eagle3_quick_check.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ pipeline:
_factory_: "slurm_factory"
nodes: 1
gpus_per_node: 4
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc2
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 2: Dump hidden states from target model
task_1:
Expand All @@ -56,7 +56,7 @@ pipeline:
_factory_: "slurm_factory"
nodes: 1
gpus_per_node: 4
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc2
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 3: Train EAGLE3 draft head (offline, single task)
task_2:
Expand All @@ -78,7 +78,7 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 4
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc2
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 4: Benchmark speculative decoding (VLLM backend)
#
Expand All @@ -90,7 +90,7 @@ pipeline:
# TRTLLM_LAUNCH_SCRIPT: trtllm-llmapi-launch
# slurm_config:
# ntasks_per_node: 4
# container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc2
# container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20
#
# To use SGLang
#
Expand Down
2 changes: 1 addition & 1 deletion tools/launcher/examples/Qwen/Qwen3-8B/hf_eagle3_dryrun.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,4 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20
6 changes: 3 additions & 3 deletions tools/launcher/examples/Qwen/Qwen3-8B/hf_offline_eagle3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ pipeline:
nodes: 1
ntasks_per_node: 8
gpus_per_node: 8
container: nvcr.io/nvidia/tensorrt-llm/release:1.2.0
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 2: Dump hidden states from target model
task_1:
Expand All @@ -61,7 +61,7 @@ pipeline:
nodes: 1
ntasks_per_node: 8
gpus_per_node: 8
container: nvcr.io/nvidia/tensorrt-llm/release:1.2.0
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 3: Train EAGLE3 draft head (offline, single task)
task_2:
Expand All @@ -79,7 +79,7 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 8
container: nvcr.io/nvidia/tensorrt-llm/release:1.2.0
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 4: Benchmark speculative decoding (VLLM backend)
task_3:
Expand Down
6 changes: 3 additions & 3 deletions tools/launcher/examples/Qwen/Qwen3-8B/hf_offline_eagle3_ptq.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ pipeline:
nodes: 1
ntasks_per_node: 4
gpus_per_node: 4
container: nvcr.io/nvidia/tensorrt-llm/release:1.2.0
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 2: Dump hidden states from target model
task_1:
Expand All @@ -62,7 +62,7 @@ pipeline:
nodes: 1
ntasks_per_node: 4
gpus_per_node: 4
container: nvcr.io/nvidia/tensorrt-llm/release:1.2.0
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 3: Train EAGLE3 draft head (offline, single task) and export checkpoint
task_2:
Expand All @@ -87,7 +87,7 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 4
container: nvcr.io/nvidia/tensorrt-llm/release:1.2.0
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 4: PTQ — quantize the EAGLE3 model using offline hidden states
task_3:
Expand Down
4 changes: 2 additions & 2 deletions tools/launcher/examples/Qwen/Qwen3-8B/hf_online_eagle3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

task_1:
script: common/eagle3/train_eagle.sh
Expand All @@ -49,7 +49,7 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 8
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

task_2:
script: common/specdec_bench/quick_check.sh
Expand Down
2 changes: 1 addition & 1 deletion tools/launcher/examples/Qwen/Qwen3-8B/hf_ptq.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,4 +48,4 @@ pipeline:
ntasks_per_node: 1
gpus_per_node: 1
time: "04:00:00"
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc7
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20
4 changes: 2 additions & 2 deletions tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 2: Streaming DFlash training — node 0 vllm serve, node 1 trainer.
# DFlash extracts 5 target layers (build_target_layer_ids(36,5)=[1,9,17,25,33], the
Expand Down Expand Up @@ -90,7 +90,7 @@ pipeline:
- MIN_ACCEPTANCE_LENGTH: "1.2"
slurm_config:
_factory_: "slurm_factory"
container: "vllm/vllm-openai:nightly"
container: vllm/vllm-openai:nightly

@coderabbitai coderabbitai Bot Jul 16, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

vLLM smoke-test containers still float on nightly, not pinned to v0.22.0.

The PR objective states vLLM is being pinned to v0.22.0, but these two workflows still reference the unpinned vllm/vllm-openai:nightly tag for their vLLM smoke-test task, making runs non-reproducible.

  • tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash.yaml#L93-L93: pin container to vllm/vllm-openai:v0.22.0 (or the multi-arch v0.22.0 image used elsewhere in this PR).
  • tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash_multi_node.yaml#L96-L96: same fix — pin to vllm/vllm-openai:v0.22.0.
📍 Affects 2 files
  • tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash.yaml#L93-L93 (this comment)
  • tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash_multi_node.yaml#L96-L96
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash.yaml` at line 93,
Pin the vLLM smoke-test container from nightly to v0.22.0 in both
tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash.yaml:93-93 and
tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash_multi_node.yaml:96-96
by updating each container reference to the v0.22.0 image, using the multi-arch
variant if that is the established image elsewhere in the PR.

@noeyy-mino noeyy-mino Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ChenhanYu @h-guo18， For the vLLM container, can we use the latest tag—i.e., vllm/vllm-openai:latest—so that it always points to the newest vLLM release version? This way, we wouldn’t need to frequently update the YAML file.

nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
4 changes: 2 additions & 2 deletions tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_dflash_multi_node.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 2: Streaming DFlash training — 2 serve replicas (TP=2) + 2 trainer nodes (2 GPU each).
# DFlash extracts 5 target layers (build_target_layer_ids(36,5)=[1,9,17,25,33], the
Expand Down Expand Up @@ -93,7 +93,7 @@ pipeline:
- MIN_ACCEPTANCE_LENGTH: "1.2"
slurm_config:
_factory_: "slurm_factory"
container: "vllm/vllm-openai:nightly"
container: vllm/vllm-openai:nightly
nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
2 changes: 1 addition & 1 deletion tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_eagle3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# capture ids = default_eagle_aux_layer_ids(36)=[1,17,32] shifted +1, plus final
# layer 36 -> [2,18,33,36].
Expand Down
2 changes: 1 addition & 1 deletion tools/launcher/examples/Qwen/Qwen3-8B/hf_streaming_eagle3_multi_node.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 2: Streaming EAGLE3 training — 2 serve replicas (TP=2) + 2 trainer nodes (2 GPU each).
# Capture ids: default_eagle_aux_layer_ids(36)=[1,17,32] +1, plus final layer 36.
Expand Down
2 changes: 1 addition & 1 deletion tools/launcher/examples/Qwen/Qwen3.5-4B/specdec_bench.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ pipeline:
- --concurrency 8
- --num_requests 80
- --output_length 4096
- --max_seq_len 40960
- --max_seq_len 65536
- --aa_timing
- --show_progress
- --save_dir /scratchspace/qwen35_4b_none_vllm/throughput_32k
Expand Down
2 changes: 1 addition & 1 deletion tools/launcher/examples/Qwen/Qwen3.5-4B/specdec_bench_mtp_vllm.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ pipeline:
- --ep_size 1
- --concurrency 8
- --num_requests 80
- --runtime_params common/specdec_bench/runtime_params_throughput_32k.yaml
- --max_seq_len 65536
- --output_length 4096
- --aa_timing
- --show_progress
Expand Down
2 changes: 1 addition & 1 deletion tools/launcher/examples/moonshotai/Kimi-K2.5/hf_dflash_dryrun.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,4 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20
2 changes: 1 addition & 1 deletion tools/launcher/examples/moonshotai/Kimi-K2.5/hf_eagle3_dryrun.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,4 +57,4 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20
2 changes: 1 addition & 1 deletion tools/launcher/examples/moonshotai/Kimi-K2.5/hf_offline_dflash.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,4 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 8
container: vllm/vllm-openai:v0.22.0-aarch64
container: vllm/vllm-openai:v0.22.0
4 changes: 2 additions & 2 deletions tools/launcher/examples/moonshotai/Kimi-K2.5/hf_streaming_dflash.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ pipeline:
ntasks_per_node: 1
# The cluster QOS requires whole-node GPU allocation though make_dataset is CPU-only.
gpus_per_node: 4
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

task_1:
script: common/eagle3/train_eagle_streaming.sh
Expand Down Expand Up @@ -96,4 +96,4 @@ pipeline:
segment: 2
ntasks_per_node: 1
gpus_per_node: 4
container: vllm/vllm-openai:v0.22.0-aarch64
container: vllm/vllm-openai:v0.22.0
4 changes: 2 additions & 2 deletions tools/launcher/examples/moonshotai/Kimi-K2.5/hf_streaming_dflash_multi_node.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ pipeline:
ntasks_per_node: 1
# The cluster QOS requires whole-node GPU alloc even though make_dataset is CPU-only.
gpus_per_node: 4
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Streaming DFlash training: 2 serve replicas (TP=4) + 2 trainer nodes.
task_1:
Expand Down Expand Up @@ -99,4 +99,4 @@ pipeline:
gpus_per_node: 4
# Pin 0.22.0: 0.22.1 regressed Kimi serve (profile_run runs the ViT and hits
# a `fmax()` crash in kimi_k25_vit.py); 0.17.0 fails SpeculativeConfig validation.
container: vllm/vllm-openai:v0.22.0-aarch64
container: vllm/vllm-openai:v0.22.0
6 changes: 3 additions & 3 deletions tools/launcher/examples/moonshotai/Kimi-K2.5/hf_streaming_eagle3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ pipeline:
ntasks_per_node: 1
# The cluster QOS requires whole-node GPU alloc (4) even though make_dataset is CPU-only.
gpus_per_node: 4
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 2: Streaming EAGLE3 training (node0 serve TP=4 / node1 train), 4 GPU/node
task_1:
Expand Down Expand Up @@ -79,7 +79,7 @@ pipeline:
segment: 2
ntasks_per_node: 1
gpus_per_node: 4
container: vllm/vllm-openai:v0.22.0-aarch64
container: vllm/vllm-openai:v0.22.0

# Step 3: Benchmark speculative decoding (VLLM backend, Kimi served at TP=4)
task_2:
Expand All @@ -103,4 +103,4 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 4
container: vllm/vllm-openai:v0.22.0-aarch64
container: vllm/vllm-openai:v0.22.0
6 changes: 3 additions & 3 deletions tools/launcher/examples/moonshotai/Kimi-K2.5/hf_streaming_eagle3_multi_node.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ pipeline:
ntasks_per_node: 1
# The cluster QOS requires whole-node GPU allocation though make_dataset is CPU-only.
gpus_per_node: 4
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

task_1:
script: common/eagle3/train_eagle_streaming.sh
Expand Down Expand Up @@ -81,7 +81,7 @@ pipeline:
gpus_per_node: 4
# Pin 0.22.0: 0.22.1 regressed Kimi serve (profile_run runs the ViT and hits
# a `fmax()` crash in kimi_k25_vit.py); 0.17.0 fails SpeculativeConfig validation.
container: vllm/vllm-openai:v0.22.0-aarch64
container: vllm/vllm-openai:v0.22.0

task_2:
script: common/specdec_bench/quick_check.sh
Expand All @@ -104,4 +104,4 @@ pipeline:
ntasks_per_node: 1
gpus_per_node: 4
# Match the serve task's pin (see task_1) — 0.22.1 broke Kimi serve.
container: vllm/vllm-openai:v0.22.0-aarch64
container: vllm/vllm-openai:v0.22.0
4 changes: 2 additions & 2 deletions tools/launcher/examples/moonshotai/Kimi-K2.5/specdec_bench.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# `prepare_data.py --dataset speed --config all`, then replace --mtbench with
# `--dataset speed` + `--dataset_path .../data/speed/<split>`.
#
# NOTE on container: pinned to the aarch64 v0.22.0 image (GB200).
# NOTE on container: multi-arch v0.22.0 tag (resolves to amd64 on x86_64, arm64 on GB200).
#
# Run ON the cluster login node (paramiko can't reach the cluster through its login proxy):
# export SLURM_HOST=localhost SLURM_ACCOUNT=<your_account> \
Expand Down Expand Up @@ -58,4 +58,4 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 4
container: vllm/vllm-openai:v0.22.0-aarch64
container: vllm/vllm-openai:v0.22.0
2 changes: 1 addition & 1 deletion tools/launcher/examples/openai/gpt-oss-20b/hf_streaming_dflash_multi_node.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 2: Streaming DFlash training — 2 serve replicas (TP=2) + 2 trainer nodes (2 GPU each).
# DFlash extracts 5 target layers (build_target_layer_ids(24,5)=[1,6,11,16,21], the
Expand Down
2 changes: 1 addition & 1 deletion tools/launcher/examples/openai/gpt-oss-20b/hf_streaming_eagle3_multi_node.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ pipeline:
nodes: 1
ntasks_per_node: 1
gpus_per_node: 1
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc10
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20

# Step 2: Streaming EAGLE3 training — 2 serve replicas (TP=2) + 2 trainer nodes (2 GPU each).
# Capture ids: default_eagle_aux_layer_ids(24)=[1,11,20] +1, plus final layer 24.
Expand Down
2 changes: 1 addition & 1 deletion tools/launcher/slurm_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ def slurm_factory(
nodes: int = 1,
ntasks_per_node: int = 1,
gpus_per_node: Optional[int] = 1,
container: str = "nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc8",
container: str = "nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc20",
modelopt_install_path: str = "/usr/local/lib/python3.12/dist-packages/modelopt",
container_mounts: list[str] = [
"{}:/hf-local".format(os.environ.get("SLURM_HF_LOCAL", "/hf-local")),
Expand Down
Loading