perf(pytorch): add opt-in torch.compile for decode CUDA graphs

[grimoire]

Motivation

LMDeploy currently captures CUDA graphs directly on the raw model. Tuned custom kernels are captured correctly, but compiler-visible PyTorch operations cannot be fused by Inductor.

This PR adds an experimental decode-only path that runs torch.compile before the existing CUDA graph capture. The goal is to fuse small PyTorch regions while preserving tuned attention, RMSNorm, fused-MoE, Triton, and other custom kernel launches inside the outer CUDA graph.

Changes

• Lazily build a compiled model callable on the first decode CUDA graph capture.
• Keep prefill, eager execution, and non-CUDA backends on the raw model.
• Use fullgraph=False and dynamic=False; graph breaks are allowed.
• Register package-level Dynamo trace rules for LMDeploy kernels, Triton, and FlashAttention wrappers.
• Disable Inductor's internal CUDA graphs because LMDeploy owns the outer capture.
• Preserve eager numerical behavior with precision-cast and division-rounding emulation.
• Raise Dynamo specialization limits to 1024 for fixed decode capture buckets.
• Keep the feature disabled by default.

Enable it with:

LMDEPLOY_ENABLE_DECODE_TORCH_COMPILE=1

No model-specific fusion patterns or model code changes are introduced.

Limitations

• This remains experimental and disabled by default.
• It uses private torch._dynamo.trace_rules APIs and has primarily been validated with PyTorch 2.10.
• Compilation increases initial CUDA graph warmup time. A persistent TORCHINDUCTOR_CACHE_DIR is recommended.
• DeepSeek-V4 execution works mechanically, but compiled and raw TP4 runs showed greedy-token drift. This PR therefore does not claim generic DSV4 correctness or readiness for default enablement.
• Prefill compilation and piecewise CUDA graphs are intentionally out of scope.

[Copilot]

Pull request overview

Adds an experimental, opt-in torch.compile path that is only used during decode CUDA graph capture on CUDA backends, aiming to fuse compiler-visible PyTorch regions while keeping existing custom-kernel behavior within the outer CUDA graph capture.

Changes:

• Introduces lazy construction of a compiled decode-only model callable and routes decode graph capture through it.
• Adds a new environment toggle LMDEPLOY_ENABLE_DECODE_TORCH_COMPILE (default off).
• Adds unit tests covering decode-only behavior, lazy compilation, non-CUDA behavior, and one-time Dynamo configuration.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
tests/pytorch/engine/test_decode_torch_compile.py	Adds unit tests for decode-only torch.compile integration and Dynamo configuration behavior.
lmdeploy/pytorch/envs.py	Adds env var parsing for LMDEPLOY_ENABLE_DECODE_TORCH_COMPILE.
lmdeploy/pytorch/backends/cuda/graph_runner.py	Implements decode-only compile configuration + lazy compiled callable usage during decode CUDA graph capture.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[RunningLeon]

Make the compile options compatible with supported PyTorch versions —
emulate_divison_rounding exists in PyTorch 2.10, but is rejected by both PyTorch 2.8 and 2.11; newer PyTorch uses eager_numerics.division_rounding. Since current main supports Torch 2.0–2.12.1, enabling LMDEPLOY_ENABLE_DECODE_TORCH_COMPILE=1 causes the first decode capture to fail immediately on common supported installations.