Add the Eagle3Speculator module for speculative decoding

[digantdesai]

Eagle3Speculator holds a shared target and an EAGLE-3 draft head and exposes
prefill, target_verify, draft_decode, and decode. It is written against the
TapTarget protocol (target.py) rather than a concrete model, so any target that
exposes EAGLE-3 hidden-state taps works; gemma4-31B is the reference, registered
in target.py's TARGETS (loader + forward-length bounds). The export lowers only
the first three methods; under the shifted (vLLM-EAGLE) runner scheme decode is
unnecessary.

The target methods return the fused (hidden-size) draft feature, so draft_decode
takes a single uniform feature whether it comes from the target (confirmed
positions) or the draft's own recurrent output (proposed positions).
target_verify returns the per-position greedy argmax target id; verify_ids[i] is
the greedy token after token i. The draft pairs target hidden_state_t with
token_{t+1}, so after verification the runner reseeds the draft cache from the
feature target_verify already produced for the accepted positions, paired with
the corrected token; the corrected/bonus token never needs its own target
forward, so the three exported methods suffice at one target forward per round.
Draft heads without their own embedding table are rejected.

Tests cover each method, target_verify reusing a prefilled cache, the
verification alignment, a cached draft chain matching the stateless recompute,
the full shifted speculative loop reproducing greedy decoding, and TapTarget
protocol conformance.

Authored with assistance from Claude Code.

[digantdesai]

Stack from ghstack (oldest at bottom):

• Add EAGLE-3 end-to-end speculative-decode test #20157
• Add the EAGLE-3 speculative-decoding runner (CUDA) #20156
• Add the EAGLE-3 speculator CUDA export #20155
• Add a weight-stationary INT4 GEMM for small M #20154
• -> Add the Eagle3Speculator module for speculative decoding #20153
• Add KV cache to the EAGLE-3 draft head #20152
• Add Gemma4 EAGLE-3 eager reference #20151
• Add Gemma4 EAGLE-3 hidden-state taps #20150
• Add EAGLE-3 draft head #20149

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20153

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 2 Pending

As of commit 66da947 with merge base dc55469 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.