Add DeepSeek 3.2 support

[michaelroyzen]

Summary

Adds Liger Kernel support for Hugging Face Transformers deepseek_v32 / DeepSeek-V3.2.

• Adds a DeepSeek-V3.2 FLCE forward path for DeepseekV32ForCausalLM.
• Wires apply_liger_kernel_to_deepseek_v32 into public exports and model-type autodispatch.
• Patches supported non-attention kernels: RMSNorm, SwiGLU for dense/shared MLPs, CrossEntropyLoss, and FusedLinearCrossEntropy.
• Leaves DeepSeek-V3.2 attention on the upstream Transformers implementation, including MLA, DSA sparse attention/indexer, SDPA dispatch, and custom RoPE handling.
• Adds unit and convergence coverage for fp32/bf16 and logits/FLCE paths.
• Updates README patching table.

Implementation Notes

DeepSeek-V3.2 uses custom attention machinery in Transformers:

• _supports_sdpa = True
• _supports_flash_attn = False

This PR therefore validates Liger support with attn_implementation="sdpa" and does not attempt to patch or replace the model's sparse attention path. FlashAttention 2 is not advertised as supported by the Transformers model implementation.

RoPE is intentionally not patched. DeepSeek-V3.2 uses interleaved partial RoPE in the main MLA path and a separate non-interleaved RoPE path for the DSA indexer, which is not compatible with Liger's generic RoPE swap.

For SwiGLU, sparse MoE routed experts are left on the upstream packed expert implementation. Liger patches the dense MLP layers and sparse-layer shared_experts, reusing the existing Qwen3 MoE-compatible SwiGLU wrapper to avoid introducing a duplicate DeepSeek-specific helper.

Validation

Validated on H200 with CUDA 13 nightly PyTorch:

• torch 2.14.0.dev20260619+cu130
• torchvision 0.29.0.dev20260620+cu130
• torch.version.cuda == 13.0

The CUDA/cuDNN wheel library directories were prepended to LD_LIBRARY_PATH for SDPA:

export LD_LIBRARY_PATH="$PWD/.venv/lib/python3.12/site-packages/nvidia/cudnn/lib:$PWD/.venv/lib/python3.12/site-packages/nvidia/cu13/lib:$LD_LIBRARY_PATH"

Passing checks:

uv run --no-sync pytest test/transformers/test_monkey_patch.py -k "deepseek_v32" -q
uv run --no-sync pytest test/convergence/bf16/test_mini_models_with_logits.py -k "deepseek_v32" -xvs
uv run --no-sync pytest test/convergence/bf16/test_mini_models.py -k "deepseek_v32" -xvs
uv run --no-sync pytest test/convergence/fp32/test_mini_models_with_logits.py -k "deepseek_v32" -xvs
uv run --no-sync pytest test/convergence/fp32/test_mini_models.py -k "deepseek_v32" -xvs
uv run --no-sync make checkstyle

Convergence results:

• bf16 logits path: passed
• bf16 FLCE path: passed
• fp32 logits path: passed
• fp32 FLCE path: passed

The fp32 FLCE eval loss matched exactly in the latest run (9.922870635986328 reference and Liger). The bf16 FLCE eval loss differed by approximately 1.1e-5, within tolerance.

[michaelroyzen]

Hi @Tcc0403 @Mecoli1219 can I get a review please?

[Tcc0403]

LGTM, cc @Mecoli1219 @vaibhavjindal for ci check

[michaelroyzen]

Hi @Mecoli1219, can we merge please?