feat(qwen3_5_mtp): Qwen3.5 / Qwen3.5-MoE MTP speculative decoding

[sufubao]

What

Adds Qwen3.5 / Qwen3.5-MoE Multi-Token Prediction (MTP / speculative decoding) end-to-end, together with the model-agnostic verify-decode machinery it needs. Builds on the BaseMTPModel refactor in #1337 (merged).

Commits

1. feat(mtp): MTP verify-decode infrastructure — model-agnostic verify dispatch in TpPartBaseModel, dedicated decode CUDA-graph capture/replay for the (mtp_step+1)-expanded verify layout, a shared mtp_verify_extra_state block, and fa3 decode attention narrowed to the verify layout (b_att_seq_len + causal) for fp / fp8 / mla.
2. feat(qwen3_5_mtp): Qwen3.5 / Qwen3.5-MoE MTP draft models — dense and MoE draft packages sharing a weight-retarget mixin (mtp.* head, embeddings shared with the main model) and the MTP pre-layer fuse.
3. feat(qwen3next): GDN spec-decode verify path + linear-att cache split — a per-sequence spec causal_conv1d kernel, a widened conv working slot split from the committed (narrow) persisted slot, and MTP draft full-attn KV-slot accounting across the linear-att cache config / mem operator / req manager.
4. feat(scheduler): MTP verify backend + accept-len transport — a single draft-model factory keyed on (model_type, mtp_mode), the verify decode batch, eagle + vanilla draft decode, and per-request accept-length (b_num_accepted_tokens) transport through the chunked-prefill and dp backends.
5. test(mtp): unit tests + static MTP benchmark — and the .gitignore benchmark-output rule anchored to /benchmark.

Testing

• pre-commit (black 21.12b0 + flake8 6.1.0) clean.
• Unit tests under unit_tests/ cover verify-extra-state, decode CUDA-graph layouts, fa3 narrowing, GDN verify equivalence, linear-att conv/SSM split + CPU-cache persistence, and the draft-model factory.

[gemini-code-assist]

Code Review

This pull request implements comprehensive multi-token prediction (MTP) and speculative decoding support for Qwen3.5 and Qwen3Next models, including updates to attention backends, CUDA graph warmup layouts, and Triton kernels for spec-decode updates. Feedback on the changes highlights a performance concern regarding synchronous device-to-host transfers when validating b_num_accepted_tokens on the GPU, as well as a potential division-by-zero error in the causal conv1d update kernel if the batch size is empty.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Calling .min() and .max() on the GPU tensor b_num_accepted_tokens and casting to int causes a synchronous device-to-host (D2H) transfer. Since this function is called on the eager decode hot path, this synchronization will stall the CPU and degrade inference performance. Consider performing this validation on the CPU side (e.g., on the list of mtp_accept_len in infer_batch.py before moving it to CUDA) to keep the execution fully asynchronous.

[gemini-code-assist]

If batch (the size of conv_state_indices) is 0, calculating seqlen = x.size(0) // batch will raise a ZeroDivisionError. To make the function robust against empty batches (which can occur in edge cases or testing), add a defensive guard to return x immediately if batch == 0.

        assert conv_state_indices is not None\n        batch = conv_state_indices.size(0)\n        if batch == 0:\n            return x\n        dim = x.size(1)