[Klaud Cold]minimaxm3-fp8-mi300x-vllm-mtp: day-zero MiniMax-M3 EAGLE3 (MTP) MI300X recipe

[functionstackx]

Summary

Adds the EAGLE3 speculative-decoding (spec-decoding: mtp) sibling of minimaxm3-fp8-mi300x-vllm (#1746): MiniMax-M3 MXFP8 on MI300X (gfx942) single-node vLLM (ROCm), pairing MiniMaxAI/MiniMax-M3-MXFP8 with the Inferact/MiniMax-M3-EAGLE3 draft head. Based on the MI300X non-MTP recipe + the (merged) MI355X MTP recipe (#1745).

New benchmark script

benchmarks/single_node/fixed_seq_len/minimaxm3_fp8_mi300x_mtp.sh, mirroring minimaxm3_fp8_mi300x.sh:

• Keeps the MI300X serve deltas vs MI355X: the default BF16 KV cache (gfx942 has no calibrated ROCm FP8 attention scales — fallback scale 1.0 corrupts accuracy) and --no-enable-prefix-caching. Same --block-size 128, --language-model-only, --attention-backend TRITON_ATTN, --enforce-eager, minimax_m3 parsers.
• Adds the EAGLE3 draft: --speculative-config '{"method":"eagle3","model":"Inferact/MiniMax-M3-EAGLE3","num_speculative_tokens":3}', draft download, --use-chat-template for realistic acceptance.
• Carries the same in-place EAGLE3 patch as the MI355X MTP recipe (validated green there): the shipped vllm/vllm-openai-rocm:minimax-m3 image's AMD MiniMax-M3 class lacks SupportsEagle3, so the recipe patches the installed amd/model.py before serving ([AI generated draft] minimax_m3(amd): implement SupportsEagle3 for EAGLE3 spec decoding on ROCm functionstackx/vllm#1, upstreamed as [Bug Fix] [MiniMax-M3] Implement EAGLE3 support on the AMD MiniMax M3 vllm-project/vllm#45546). Idempotent; hard-fails on base drift. The image's amd/model.py is shared across MI300X/MI355X (same ROCm image), so the patch anchors match identically — dry-run verified locally.

Config (amd-master.yaml)

minimaxm3-fp8-mi300x-vllm-mtp, same image and mi300x runner. TP8-only search space (gfx942 192 GB is memory-tight, like H100), with TP8 latency rows started at conc 1 (matching the H100/MI355X MTP recipes):

• 1k1k: TP8 (1–128), TP8+EP8 (256)
• 8k1k: TP8 (1–64), TP8+EP8 (128–256)

Launcher fix

launch_mi300x-amds.sh hardcoded _mi300x.sh with no SPEC_SUFFIX — an mtp config would silently run the non-MTP script. Added SPEC_SUFFIX (same fix applied to the H100 launchers), so mtp routes to minimaxm3_fp8_mi300x_mtp.sh.

Validation

• generate_sweep_configs.py test-config → 18 configs, all spec-decoding=mtp on mi300x, min conc 1, scenario-trimmed max-model-len 2304 / 9472.
• bash -n passes on the script + launcher; embedded patch dry-run applies cleanly to the image's amd/model.py and ast.parses; launcher routing simulated (mtp → _mtp.sh).

Like #1745, this is a validation harness (runtime monkey-patch); once the upstream fix is in a rebuilt ROCm image, the in-place patch idempotently no-ops.

🤖 Generated with Claude Code

---

Note

Medium Risk
Benchmark-only changes, but the recipe mutates installed vLLM model code at runtime; drift in the image could fail jobs or mis-route MTP sweeps if launcher routing regresses.

Overview
Adds MiniMax-M3 MXFP8 EAGLE3 speculative decoding on MI300X as the spec-decoding: mtp sibling of the existing non-MTP MI300X recipe, pairing MiniMaxAI/MiniMax-M3-MXFP8 with Inferact/MiniMax-M3-EAGLE3 (3 speculative tokens).

A new sweep config minimaxm3-fp8-mi300x-vllm-mtp in amd-master.yaml keeps a TP8-only search space (memory-tight gfx942), with TP8 latency sweeps starting at conc 1 and high-concurrency TP8+EP8 rows for 1k1k and 8k1k.

The new script minimaxm3_fp8_mi300x_mtp.sh serves with EAGLE3 --speculative-config, downloads the draft model, uses chat-template prompts for realistic acceptance, and retains MI300X-specific choices (BF16 KV, TRITON_ATTN, no prefix caching). Because the shipped ROCm image’s AMD MiniMax-M3 model lacks SupportsEagle3, the recipe patches installed amd/model.py in-place before vllm serve (idempotent, aborts on anchor drift).

launch_mi300x-amds.sh now sets SPEC_SUFFIX when SPEC_DECODING=mtp so MTP configs run *_mtp.sh instead of silently using the base script. perf-changelog.yaml documents the new config key.

^{Reviewed by Cursor Bugbot for commit 36265ae. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27487999086
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27487999086

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27487999691
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27487999691

[functionstackx]

/reuse-sweep-run

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27489883815
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27489883815