Add minimax M3 MXFP8 MI355X vLLM EAGLE3 (related PR for upstreaming patch https://github.com/vllm-project/vllm/pull/45546)

[functionstackx]

Summary

PR for upstreaming patch vllm-project/vllm#45546

Test PR to validate the MiniMax-M3 EAGLE3 fix on real MI355X hardware before the ROCm image is rebuilt. (Re-opened from #1744, which hit a GitHub Actions glitch.) Built on the EAGLE3 MI355X recipe (60d9910f).

Background

EAGLE3 MTP on MI355X failed engine init (RuntimeError: Model does not support EAGLE3 interface but aux_hidden_state_outputs was requested). Root cause: the MiniMax-M3 impl is platform-split, and the AMD model class (vllm/models/minimax_m3/amd/model.py) doesn't implement SupportsEagle3, while the NVIDIA one does — which is why B300/B200/H100/H200 pass and MI355X doesn't. Fix authored upstream as a draft on the fork: functionstackx/vllm#1.

What this PR does

The recipe applies that fix in-place to the installed vLLM inside the container, immediately before vllm serve:

• Adds EagleModelMixin to the inner MiniMaxM3Model + aux-hidden-state emission in forward(), and SupportsEagle3 to MiniMaxM3SparseForCausalLM / MiniMaxM3SparseForConditionalGeneration — mirroring nvidia/model.py.
• Idempotent (skips if already patched) and hard-fails if the installed amd/model.py has drifted from the expected base, so we never silently run unpatched and mislabel the result.
• Verified locally: the image's amd/model.py (commit g4a560dd8d) is byte-identical to the patch base, all 5 anchors match exactly, the patched source ast.parses, and the embedded patcher applies + is idempotent on a sandboxed copy of the image file.

Serve config unchanged from the EAGLE3 recipe: --speculative-config '{"method":"eagle3","model":"Inferact/MiniMax-M3-EAGLE3","num_speculative_tokens":3}', TRITON_ATTN, --enforce-eager, block-size 128.

Expected outcome

Green → the upstream patch is correct and the held #1742 recipe just needs that patch in the image. This PR is a validation harness, not meant to merge as-is (the runtime monkey-patch should be replaced by a rebuilt image).

🤖 Generated with Claude Code

---

Note

Medium Risk
The recipe mutates installed vLLM model code at runtime; drift or patch failure can break jobs or skew results, though it fails loudly on mismatch. Scope is benchmark/CI only, not production serving.

Overview
Adds MI355X EAGLE3 speculative-decoding coverage for MiniMax-M3 MXFP8 vLLM: a new minimaxm3-fp8-mi355x-vllm-mtp CI config and minimaxm3_fp8_mi355x_mtp.sh, plus a perf-changelog entry.

The config pairs MiniMaxAI/MiniMax-M3-MXFP8 with draft Inferact/MiniMax-M3-EAGLE3 (spec-decoding: mtp, 3 tokens). Search space follows the non-MTP MI355X recipe but caps high concurrency and drops tp2-ep2, aligned with other MiniMax-M3 MTP entries.

The shell recipe serves with EAGLE3 --speculative-config, downloads the draft model, uses TRITON_ATTN (no CUDA-style drafter backend override), and runs benchmarks with --use-chat-template. Because the pinned ROCm image’s AMD minimax_m3 model lacks SupportsEagle3, the job patches installed vllm in-place before vllm serve (idempotent, aborts if anchors drift)—intended as a hardware validation harness until upstream/image picks up the fix.

^{Reviewed by Cursor Bugbot for commit 5d1ddae. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27477411944
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27477411944

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 149e11e. Configure here.}

[cursor]

Draft download lacks NFS retry

Medium Severity

The recipe fetches the unstaged Inferact/MiniMax-M3-EAGLE3 draft with a single hf download into the shared NFS HF cache, while the MI355X launcher mounts that cache for MiniMax-M3 runs. Parallel matrix jobs can contend on hub lock files the same way as other MiniMax MTP recipes on network storage, but this script omits the retry loop those siblings use for the draft.

 

^{Reviewed by Cursor Bugbot for commit 149e11e. Configure here.}

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27477412884
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27477412884

[functionstackx]

/reuse-sweep-run

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27480644316
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27480644316