Add the EAGLE-3 speculator CUDA export

[digantdesai]

Exports a registered target (--target-model; gemma4-31B is the reference, see
target.py) and an EAGLE-3 draft head to a single .pte with three methods lowered
together via the CUDA (AOTI) backend: prefill and target_verify (sharing the
target KV cache) and draft_decode (the draft KV cache). The load and the target's
forward-length bounds (min/max tokens in one forward) come from its TargetSpec,
not gemma-specific code; a standalone target decode is omitted because the
shifted runner reseeds from target_verify's feature.

The model stays on the host during export so AOTInductor streams weights to the
GPU per kernel, keeping peak device memory low for the INT4 31B target. For the
static chain_len+1 verify forward to use the small-M INT4 GEMM, the export raises
int4_dispatch.MATVEC_MAX_M to 8 for this export only and restores it afterward;
prefill's dynamic length is then exported with min = max(min_forward_len, 9) and
published as get_min_prefill_chunk (the runner enforces it; no chunking). The
verify window (chain_len+1) is validated against the target's min/max forward
length and the GEMM limit.

Also slices instead of chunks in the draft's rotary helper: chunk lowers to
aten::split_copy, which the AOTI CUDA backend has no fallback kernel for.

Authored with assistance from Claude Code.

[digantdesai]

Stack from ghstack (oldest at bottom):

• Add EAGLE-3 end-to-end speculative-decode test #20157
• Add the EAGLE-3 speculative-decoding runner (CUDA) #20156
• -> Add the EAGLE-3 speculator CUDA export #20155
• Add a weight-stationary INT4 GEMM for small M #20154
• Add the Eagle3Speculator module for speculative decoding #20153
• Add KV cache to the EAGLE-3 draft head #20152
• Add Gemma4 EAGLE-3 eager reference #20151
• Add Gemma4 EAGLE-3 hidden-state taps #20150
• Add EAGLE-3 draft head #20149

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20155

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 5 Pending

As of commit 480a068 with merge base dc55469 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.