Add the EAGLE-3 speculative-decoding runner (CUDA)

[digantdesai]

A C++ runner that drives the speculator .pte with the shifted (vLLM-EAGLE)
scheme: the draft pairs the target hidden state at position t with token t+1, so
each round runs one target forward (target_verify) and reseeds the next draft
chain from the hidden states verify already produced -- no standalone target
decode. Greedy verification keeps output identical to greedy target decoding.
target_verify runs on stable input buffers and can be captured as a CUDA graph.

It requires the .pte metadata (fails loudly if absent) and enforces the exported
prefill range [get_min_prefill_chunk, get_max_prefill_chunk] (no chunking). The
prefill bonus token is always emitted; the speculative loop runs only when more
tokens are requested, the bonus was not EOS, and a K-token verify window fits
within get_max_seq_len (so a one-token or near-context request returns without
seeding the draft). The chat template and stop tokens are flags defaulting to
Gemma 4 IT (--chat_prefix/--chat_suffix/--stop_ids/--stop_token, --bos_id -1 to
skip) so other target/tokenizer pairs run without code changes. Device-to-host
reads are error-checked; the printed tau excludes the free prefill token.

Authored with assistance from Claude Code.

[digantdesai]

Stack from ghstack (oldest at bottom):

• Add EAGLE-3 end-to-end speculative-decode test #20157
• -> Add the EAGLE-3 speculative-decoding runner (CUDA) #20156
• Add the EAGLE-3 speculator CUDA export #20155
• Add a weight-stationary INT4 GEMM for small M #20154
• Add the Eagle3Speculator module for speculative decoding #20153
• Add KV cache to the EAGLE-3 draft head #20152
• Add Gemma4 EAGLE-3 eager reference #20151
• Add Gemma4 EAGLE-3 hidden-state taps #20150
• Add EAGLE-3 draft head #20149

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20156

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit 5725fee with merge base dc55469 ():

NEW FAILURES - The following jobs have failed:

• Lint / lintrunner (gh)
  >>> Lint for examples/models/eagle3/main.cpp:
• pull / test-arm-backend-no-driver (test_pytest_ops_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 95db882def0ce3fdf7aaa1c2671763e165d0cc0c94347dc2517bf8b130ad0386 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.