[Feature] Add GLM-5.3-Flash F6 core: text model + MTP + compose model

[jayhenry]

Stack (bottom to top):

1. [Feature] Add GLM-5.3-Flash F0: 25B cropped reference checkpoint builder #2105 feat/glm53flash-materialize-full-f0 → main
2. [Feature] Add GLM-5.3-Flash F3: Kimi Delta Attention (KDA) #2106 feat/glm53flash-f3-kda → feat/glm53flash-materialize-full-f0
3. [Feature] Add GLM-5.3-Flash F4: mHC four-stream residual #2107 feat/glm53flash-f4-mhc → feat/glm53flash-f3-kda
4. [Feature] Add GLM-5.3-Flash F5: NoPE DSA + KPool indexer + clamped SwiGLU #2108 feat/glm53flash-f5-nope-dsa → feat/glm53flash-f4-mhc
5. [Feature] Add GLM-5.3-Flash F1: VL data preprocessing pipeline #2109 feat/glm53flash-f1-vl-data → feat/glm53flash-f5-nope-dsa
6. [Feature] Add GLM-5.3-Flash F2: vision tower + projector (eager) #2110 feat/glm53flash-f2-vision-tower → feat/glm53flash-f1-vl-data
7. [Feature] Add GLM-5.3-Flash F6 core: text model + MTP + compose model #2111 feat/glm53flash-f6-text-moe → feat/glm53flash-f2-vision-tower ← you are here

Base is #2110's branch (layer 6). Review only this PR's own diff.

---

Summary

Stack layer 7/7 (top) of GLM-5.3-Flash support (base: layer 6, F2 vision tower). Covers the F6 text model + MTP + compose model, and the multimodal enablement that makes VL SFT actually runnable: sequence-parallel splice, and the VL training config/launcher.

Implements the core of the F6 milestone: the 45-layer KDA/NoPE-DSA text model with mHC four-stream residual, MTP, and the compose model wiring vision_tower + multi_modal_projector + language_model with image/video splice.

• xtuner/v1/model/moe/glm53/glm53.py: Glm53TextMoEConfig/Glm53TextMoE. Layer schedule ([KDA,KDA,KDA,DSA]x11 + KDA) read from the real checkpoint's text_config.layer_types. mHC's four-stream residual is expanded/collapsed once at the decoder-stack boundary, not per layer, so MoE._forward's aux-loss/router bookkeeping is reused unmodified. MTP reuses the base MTPLayer/MTPBlock unmodified — the design doc's originally-planned mtp.py deliverable turned out unnecessary.
• xtuner/v1/model/compose/glm53/modeling_glm53.py + Glm53BaseConfig: Glm53ForConditionalGeneration. Image/video splice uses the global mm_token_type_ids, never input_ids==video_token_id. A placeholder-count mismatch raises immediately (design doc §16.2) — unlike Qwen3-VL's compose path, this never silently continues training on a corrupted splice.
• Registers the "glm5.3" chat template / tokenize-fn branch, get_model_config_from_hf dispatch, and the SFT training config/launch script (examples/v1/config/sft_glm53.py, sft_glm53_tiny.sh) for the 8-GPU end-to-end smoke suite.

Root-caused four bugs found by end-to-end gradient-flow and 8-GPU training smoke tests (minimal repros, not assumed):

1. FLA's chunk_kda Triton kernel silently drops backward gradients when head_dim < 16 (a tl.dot minimum-K constraint) — not a production issue since real GLM-5.3-Flash uses head_dim=128.
2. DSA indexer showing no gradient with freeze_dsa_indexer=False traced to an existing, intentional torch.no_grad() wrap (not a bug).
3. torch.compile traced KDA's Triton-kernel-heavy forward under a strict boundary and hit disabled-function errors; fixed with GLM53_MOE_{NON_EP,EP}_COMPILE_CFG. Surfaced a separate real upstream limitation: FLA's prepare_chunk_indices/prepare_lens do a .tolist()-driven Python loop over cu_seqlens, incompatible with dynamo's dynamic-shape tracing (_mark_dynamic on the packed boundaries). Fixed by wrapping FLA's chunk/recurrent-kernel and short-conv entry points in torch._dynamo.disable() so dynamo graph-breaks at the call site instead (layer 2/7, KDA) — MODEL_COMPILE=1 now verified end-to-end, see the table below.
4. Under ep_size>1, KDA's o_norm (FLA's FusedRMSNormGated) crashed with a Triton illegal-memory-access — its inherited forward reads self.weight directly, which stays a DTensor under EP. Fixed with a minimal forward override that unshards weight/bias first.

Known gaps (recorded explicitly, not silently skipped): Vision SP remains the gap the previous layer already recorded; the compose layer's FSDP2 wiring has no dedicated multi-GPU parity test beyond the smoke suite below.

Test Plan

• tests/model/test_glm53_text_moe.py::TestGlm53TextMoEAccuracy::test_fsdp_accuracy: XTuner forward loss vs real transformers.Glm5NextForConditionalGeneration on the F0 25B cropped checkpoint, for (dispatcher, ep_size) in {(None, 1), ("all2all", 4), ("all2all", 8)} — all pass. The ep_size=4/8 cases double as the regression test for bug 4 above.
• sft_glm53_tiny.sh: single-node 8-GPU end-to-end SFT, real F0 checkpoint, PACK_MAX_LENGTH=16384, TOTAL_STEP=20. Default profile plus five combos (SP_SIZE=2 EP_SIZE=4; EP_SIZE=8 SP_SIZE=1; XTUNER_ACTIVATION_OFFLOAD=0; FP8=1; MODEL_COMPILE=1) all complete 20 steps with loss monotonically decreasing, no NaN/OOM. See the table below.

Text end-to-end regression (sft_glm53_tiny.sh, single-node 8-GPU, real F0 25B cropped checkpoint)

PACK_MAX_LENGTH=16384, TOTAL_STEP=20. All six profiles complete 20 steps, loss monotonically decreasing, no NaN/OOM. Numbers are rank0; mem is max_memory / reserved_memory (GB); tgs/seqlen_tgs/exp_tgs are step-20 throughput (tokens/gpu/s); time is step1's own duration (includes one-time warmup/compile) plus the step1→step20 wall-clock gap:

profile	step1 loss	step20 loss	grad_norm@20	mem@20 (GB)	tgs@20	seqlen_tgs@20	exp_tgs@20	time
default (EP4 SP1 offload=1)	11.30479	10.31409	48.41	86.22 / 106.10	12203.1	12224.0	4212.1	76s
SP2 EP4	11.31528	10.32397	48.40	75.15 / 96.00	9631.7	9631.7	2617.5	61s
EP8 SP1	11.30479	10.31380	48.18	100.44 / 112.75	11865.9	11886.2	4692.9	68s
offload=0 (EP4 SP1)	11.30479	10.31401	48.41	85.58 / 105.69	12315.3	12336.4	4944.9	64s
FP8=1 (EP4 SP1 offload=1)	11.30908	10.18042	45.88	84.73 / 104.35	6159.6	6170.2	1871.5	174s
MODEL_COMPILE=1 (EP4 SP1 offload=1)	11.30469	10.31412	48.39	82.15 / 103.28	5168.4	5177.2	878.1	371s

• SP2's per-GPU effective seqlen is halved (8192 vs 16384), so tgs is lower and wall-clock per step is also shorter than the default despite the added SP communication.
• offload=0 is faster with higher tgs than the default (offload=1) — expected, since activation CPU-GPU transfer is skipped.
• EP8 tracks the default (EP4) on loss/tgs; higher mem is because dp_size = world_size/ep_size drops from 2 to 1, so FSDP shards non-expert params over fewer dp ranks.
• default/EP8/offload=0 step1/step20 loss match to 1e-4, as expected — these knobs change parallelism/memory strategy, not the math.
• FP8=1's step20 loss differs from the bf16 baseline (10.18 vs 10.31) because FP8 genuinely changes the numerics on the (unquantized-kv_b_proj) absorbed-MLA path, not a bug; its tgs is roughly half the baseline's at this 25B/20-step smoke scale, consistent with per-tensor quantize/dequantize overhead not yet amortized.
• MODEL_COMPILE=1's step1 duration (299s) is Dynamo/Inductor graph tracing, a one-time cost; step20 loss (10.31412) matches the eager baseline (10.31409) to 1e-4, confirming compile doesn't change the math. tgs@20 is still below eager at this step count — few steps to amortize the compiled-kernel warmup, and several DSA/mHC regions stay fullgraph=False.

---

Multimodal enablement

Sequence parallel in the compose model

The compose model asserted sequence_parallel_mesh was None or size 1, so VL training could not use SP at all — the last piece of the Vision SP gap, now that the tower (layer 6) shards patches merge-aligned.

The splice is the only part that needs to know about SP. input_ids and mm_token_type_ids arrive already sharded (they are split together), so the local mask is what indexes the local embeddings; what the local mask cannot say is which global features belong to this rank. _local_feature_slice gathers the mask to compute this rank's offset — the number of same-modality placeholders on all preceding ranks — and _gather_visual_features reassembles the globally-ordered features, trimming the tower's merge-alignment padding. The gather is the autograd-aware one, so its backward reduce-scatters and hands each rank the gradient of exactly the shard it produced.

The placeholder↔feature count check now compares global totals, so a corrupted sample raises identically with and without SP. The pure-text dummy visual forward stays replicated: its gradient contribution is exactly zero, so sharding it would only add collectives.

VL training entry point

Glm53BaseConfig.from_hf raised NotImplementedError, so the VL model could not be built from a checkpoint at all. It now reads the checkpoint's single vision_config and fans it out to both XTuner modules — tower and projector are one HF module that §5.2 splits in two — and delegates the text half to Glm53TextMoEConfig.from_hf.

examples/v1/config/sft_glm53_vl.py + sft_glm53_vl_tiny.sh mirror the text pair knob for knob, so both modalities run the same acceptance matrix. Media comes from the ci_vl corpus (image + video samples). FP8 stays on the language tower only, matching how the published checkpoint stores the vision half in bf16.

hf_config intentionally remains None: _write_hf_index_and_config then copies the source checkpoint's config/tokenizer into the export, which is self-consistent for a fine-tune that does not change the architecture.

VL end-to-end regression (sft_glm53_vl_tiny.sh, single-node 8-GPU, real F0 25B cropped checkpoint)

PACK_MAX_LENGTH=16384, TOTAL_STEP=20, image + video data. All six profiles complete 20 steps, loss decreasing, no NaN/OOM. Numbers are rank0; mem is max_memory / reserved_memory:

profile	step1 loss	step20 loss	grad_norm@20	mem@20 (GB)	tgs@20	seqlen_tgs@20	exp_tgs@20	time	img_tokens
default (EP4 SP1 offload=1)	12.34349346	10.23751831	32.69	86.11 / 107.45	3479.8	7083.2	1286.7	53s	29736
SP2 EP4	12.13223362	10.24192142	52.60	73.86 / 96.08	2253.5	4721.4	1708.6	33s	6516
EP8 SP1	12.34349346	10.23978424	39.65	96.77 / 117.13	3399.5	6919.8	1183.9	56s	29736
offload=0 (EP4 SP1)	12.34349346	10.25128174	32.17	85.07 / 107.21	3514.9	7154.6	1264.8	52s	29736
FP8=1	12.34872818	9.99082375	23.11	84.92 / 106.38	3391.6	6903.8	1359.4	53s	29736
MODEL_COMPILE=1	12.34103584	10.24398804	32.14	81.74 / 104.83	4353.5	8861.7	668.4	71s	29736

• default / EP8 / offload=0 step-1 loss is bit-identical (12.34349346) — these knobs change parallelism and memory strategy, not the math, the same conclusion as the text matrix.
• SP2's loss is not comparable row-to-row: GLOBAL_BATCH_SIZE = world/sp, so it consumes different samples per step (img_tokens 6516 vs 29736). Its comparable quantity is the step-20 value, 0.004 from the default. Vision SP correctness rests on the 2-GPU bitwise parity tests in layer 6 and TestGlm53ComposeSequenceParallel here, not on this smoke.
• MODEL_COMPILE=1 gives the best throughput (+25% tgs) and the lowest peak memory, paying for it in step-1 graph compilation (71s total vs 53s).
• FP8=1's loss is visibly lower (9.99 vs 10.24), the same direction as text-side FP8: the absorbed-MLA path's numerics genuinely change; not a bug.
• EP8 peaks highest because dp_size = world_size/ep_size drops from 2 to 1, so FSDP shards non-expert params over fewer dp ranks.

VL unit coverage

TestGlm53ComposeSequenceParallel::test_image_splice_under_sp_matches_non_sp (2 GPU): two placeholder spans placed so one lands on each rank's shard — forcing the cross-rank feature slicing — and each rank's logits match the corresponding slice of the non-SP run. This replaces the guard test that asserted SP was refused. tests/model/test_glm53_compose.py 6/6.