[GLM-5.2] Preserve HF config compatibility for vLLM

.claude/skills/check-hf-config-save/SKILL.md

@@ -0,0 +1,236 @@
+---
+name: check-hf-config-save
+description: Implement a missing XTuner Hugging Face config export from the official HF model when possible, then validate it against the installed Transformers round-trip and versioned inference-engine field contracts. Use when adding a model, when hf_config is missing or returns None, when changing from_hf/hf_config/save_hf, when reviewing config.json differences, or when debugging vLLM/SGLang checkpoint-load or inference failures. Also use for requests named check_hf_config_save.
+---
+
+# Check HF Config Save
+
+## Overview
+
+First ensure that a model with an official built-in Transformers config has an
+inverse `from_hf <-> hf_config` mapping. If that export is missing, implement it
+from the official HF model before using `xtuner._testing.check_hf_config_save`
+to test the real public `from_hf -> save_hf` path. Separate changes forced by
+the installed Transformers version from fields dropped by XTuner, then protect
+fields that exact inference engine versions use for model construction or
+weight loading.
+
+## Workflow
+
+### 1. Fix the scope and version matrix
+
+1. Identify the source HF model directory, XTuner config class, export path, and
+   any engine/version named by the user or failure log.
+2. Record the executable environment versions for Python, Transformers, and any
+   installed engines. Use the requested environment; otherwise follow the repo's
+   default environment instructions.
+3. A user-specified or production-log version takes precedence. If a component
+   version is unspecified, look up the latest stable release from its official
+   PyPI project/API or official release page and inspect the matching source tag.
+   By default audit both vLLM and SGLang.
+4. List every checked version in the result. Distinguish an installed runtime
+   test from a static audit of an exact source tag.
+
+Use a table with at least these columns:
+
+| Component | Version | How selected | Validation |
+|---|---:|---|---|
+| Transformers | exact version | active environment | executable round-trip |
+| vLLM | exact version | user/log/latest official | runtime or exact-tag source audit |
+| SGLang | exact version | user/log/latest official | runtime or exact-tag source audit |
+
+Do not write `latest` without resolving it to an exact version and source.
+
+### 2. Implement a missing HF config export
+
+Exercise the supported model's public conversion path first:
+
+```python
+config = get_model_config_from_hf(SOURCE_HF_DIR)
+exported_config = config.hf_config
+```
+
+`exported_config is None`, or the corresponding public `config.save_hf(...)`
+raising the base missing-`hf_config` `NotImplementedError`, means the config-only
+export is not implemented. Classify the official model before changing code:
+
+```mermaid
+flowchart TD
+    A["Official HF model and exact revision"] --> B{"Built-in official Transformers config?"}
+    B -->|Yes| C{"XTuner hf_config implemented?"}
+    C -->|No| D["Implement inverse from_hf and hf_config mapping"]
+    C -->|Yes| E["Continue round-trip validation"]
+    D --> E
+    B -->|No; trust_remote_code only| F["Keep hf_config as None and validate source-config copying"]
+```
+
+For a built-in official Transformers config:
+
+1. Read the official checkpoint's raw `config.json` and resolve its exact
+   `model_type`, `architectures`, repository revision, and official config
+   implementation. Load it with the selected Transformers version and record
+   the concrete `PretrainedConfig` subclass. Do not use a third-party model
+   implementation as the reference.
+2. Implement or complete `from_hf` with that official class. Map every
+   architecture value needed by XTuner, use `getattr` for genuinely optional
+   versioned fields, and use `RopeParametersConfig.from_hf_config` for RoPE.
+3. Implement `hf_config` by constructing the same official config class from
+   the XTuner config's current values. It must be the inverse of `from_hf`, not
+   a cached copy of the source object. Re-emit every field consumed by
+   `from_hf`, plus raw compatibility fields required by vLLM or SGLang even when
+   Transformers treats them as optional or legacy.
+4. Preserve the official `model_type` and architecture name. Do not substitute
+   a generic `PretrainedConfig`, duplicate the official config class in XTuner,
+   or return a hand-written dictionary.
+5. Verify through public behavior that `config.hf_config` has the expected
+   official type and that `config.save_hf(...)` succeeds, then continue with
+   the helper below.
+
+If the official repository only works through `trust_remote_code` and the
+selected Transformers versions have no built-in config class, do not invent an
+`hf_config` implementation. Keep `hf_config=None`, retain the source `_hf_path`,
+and test the model's public HF save path that copies the original config,
+tokenizer, and required remote-code files. If the entire model or dispatch is
+not yet supported by XTuner, follow `$add_hf_model`; this skill only fills a
+missing exporter for an otherwise supported model.
+
+### 3. Establish the Transformers reference
+
+Compare three raw JSON states:
+
+```mermaid
+flowchart LR
+    A["Source config.json"] --> B["Current Transformers load + save"]
+    A --> C["XTuner from_hf + save_hf"]
+    B --> D["Expected serialized reference"]
+    C --> E["XTuner export"]
+    D --> F["Public helper comparison"]
+    E --> F
+```
+
+- Read the source `config.json` before `AutoConfig` normalization.
+- Load and save it directly with the active Transformers version. This is the
+  serialized reference and captures forced defaults or `__post_init__` changes.
+- Build through XTuner's public `get_model_config_from_hf`/`from_hf` API and call
+  the public `save_hf` API.
+- Compare the Transformers reference with the XTuner export. Do not use direct
+  source-versus-export equality as the primary assertion: it misclassifies
+  Transformers normalization as an XTuner bug.
+
+Inspect the exact Transformers config implementation, including generated or
+modular source and `__post_init__`, for every source-to-reference difference.
+Typical examples are derived `head_dim`, generated `layer_types`, new defaults,
+and serializer metadata, but never assume these examples cover a new model.
+
+### 4. Audit inference-engine dependencies
+
+For every changed, missing, newly defaulted, or architecture-selecting field:
+
+1. Search the exact engine tag, not an arbitrary installed or main-branch copy.
+2. Follow model registration, config access, module construction, and the weight
+   loader. Search aliases, `getattr` defaults, direct attribute access, and tensor
+   name/shape conditions.
+3. Classify the use as one of:
+   - module/parameter registration;
+   - checkpoint key or tensor-shape selection;
+   - layer topology or MoE routing;
+   - attention/RoPE behavior;
+   - ignored or default-compatible.
+4. Encode every value-sensitive dependency as `HFConfigFieldDependency`, with
+   exact engine version, JSON pointer, expected value, reason, and an official
+   source permalink.
+
+A field can be HF-equivalent yet engine-critical. For example, an engine may
+register a checkpoint parameter only when a legacy routing field has one exact
+value; omitting that field then becomes a real weight-loader failure.
+
+Static source inspection proves the dependency but not end-to-end engine
+compatibility. Run an engine checkpoint-load smoke test when the exact runtime
+and required hardware are available. Otherwise report `exact-tag source audit`
+and do not claim runtime success. Follow the repo GPU-lock instructions before
+any local GPU run.
+
+### 5. Add the model regression test
+
+Delete narrow hand-written assertions that duplicate this contract, then add a
+model test on the real public conversion path:
+
+```python
+import transformers
+
+from xtuner._testing import HFConfigFieldDependency, check_hf_config_save
+from xtuner.v1.model import get_model_config_from_hf
+
+
+def test_save_hf_matches_transformers_and_engine_contracts():
+    config = get_model_config_from_hf(SOURCE_HF_DIR)
+    assert isinstance(config.hf_config, OFFICIAL_HF_CONFIG_CLASS)
+    report = check_hf_config_save(
+        config,
+        SOURCE_HF_DIR,
+        engine_dependencies=(
+            HFConfigFieldDependency(
+                engine="vllm",
+                version="<exact-version>",
+                path="/<json-field>",
+                expected="<required-value>",
+                reason="<construction or loader dependency>",
+                source="<official exact-tag permalink>",
+            ),
+        ),
+    )
+
+    assert report.transformers_version == transformers.__version__
+    assert report.checked_engine_versions == ("vllm==<exact-version>",)
+```
+
+The helper performs two independent checks:
+
+- XTuner export matches the active Transformers direct round-trip.
+- Exported values satisfy the declared inference-engine contracts, even when
+  Transformers itself does not consume those fields.
+
+Use `allowed_export_differences={"/json/pointer": "specific reason"}` only for
+an intentional XTuner difference that is not already an engine dependency.
+Every exception needs a non-empty model-level reason.
+
+### 6. Prove the regression and run the matrix
+
+1. Run the helper's own behavior tests.
+2. Run the generated model test in the project's pinned Transformers version.
+3. Run it in each user-requested/current upgraded Transformers environment.
+4. If the old broken export is available, pass its `config.json` through the
+   same helper and show that the expected missing/extra paths fail. This is the
+   minimal proof that the test catches the original bug.
+5. Run formatting and the smallest relevant model test suite.
+
+Report:
+
+- whether `hf_config` already existed, was implemented from which official
+  config class/revision, or correctly remained `None` for remote code;
+- source-to-Transformers normalization paths;
+- Transformers-reference-to-XTuner paths (normally none, apart from documented
+  allowed contracts);
+- engine dependency, expected value, exact version, and effect;
+- which checks were executable and which were source-only.
+
+## Guardrails
+
+- Compare raw JSON, not only `AutoConfig` attributes; unknown compatibility
+  fields may disappear from a newer config class while engines still read them.
+- Exercise public APIs and real conversion behavior. Do not mock XTuner model
+  internals.
+- Do not replace the helper with a blanket list of expected keys.
+- Do not silently refresh an engine version in a test. Re-audit its exact source
+  before updating the version and permalink.
+- HF semantic equivalence is not evidence of vLLM/SGLang compatibility.
+- Do not set `hf_config=None` for a model with a built-in official config merely
+  to bypass config reconstruction or round-trip failures.
+- Preserve unrelated worktree changes and keep the implementation model-agnostic.
+
+## Completion criteria
+
+The work is complete only when the official config export path exists where it
+should, the public round-trip matches Transformers, every audited engine
+contract is preserved, and the report identifies the exact official model
+revision and component versions used.

.claude/skills/check-hf-config-save/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Check HF Config Save"
+  short_description: "Implement and validate exported HF configs"
+  default_prompt: "Use $check-hf-config-save to implement any missing official Hugging Face config export and validate it across Transformers and inference engines."

tests/model/test_glm52_moe.py

@@ -1,6 +1,7 @@
 """GLM-5.2 配置、HF 转换、路由、checkpoint 与并行数值行为测试。
 
 TestGlm52Config
+    test_save_hf_matches_transformers_and_engine_contracts: HF round-trip 与推理引擎字段契约一致。
     test_from_hf_preserves_glm_specific_behavior: HF 配置转换保留 DSA、router 与 MTP 语义。
     test_rejects_shared_physical_mtp_indexer: 非法的 physical MTP indexer 计划会被拒绝。
 TestGlm52CheckpointConversion
@@ -22,8 +23,9 @@
 import torch
 import torch.distributed as dist
 
+import transformers
 from transformers.models.glm_moe_dsa import GlmMoeDsaConfig as HFGlmMoeDsaConfig
-from xtuner._testing import DeterministicDDPTestCase
+from xtuner._testing import DeterministicDDPTestCase, HFConfigFieldDependency, check_hf_config_save
 from xtuner.v1.data_proto import SequenceContext
 from xtuner.v1.loss.ce_loss import CELossConfig
 from xtuner.v1.model import Glm52MoEConfig, get_model_config, get_model_config_from_hf
@@ -81,6 +83,55 @@ def _tiny_glm52_config() -> Glm52MoEConfig:
 
 
 class TestGlm52Config:
+    def test_save_hf_matches_transformers_and_engine_contracts(self):
+        # 走公共 from_hf/save_hf 路径，并将当前 Transformers 的强制归一化与 XTuner 丢字段区分开。
+        config = get_model_config_from_hf(GLM5_2_TINY_MOE_PATH)
+        report = check_hf_config_save(
+            config,
+            GLM5_2_TINY_MOE_PATH,
+            engine_dependencies=(
+                HFConfigFieldDependency(
+                    engine="vllm",
+                    version="0.26.0",
+                    path="/topk_method",
+                    expected="noaux_tc",
+                    reason=(
+                        "vLLM only registers gate.e_score_correction_bias for noaux_tc; "
+                        "otherwise loading the GLM checkpoint raises KeyError."
+                    ),
+                    source=(
+                        "https://github.com/vllm-project/vllm/blob/v0.26.0/"
+                        "vllm/model_executor/models/deepseek_v2.py#L314-L319"
+                    ),
+                ),
+                HFConfigFieldDependency(
+                    engine="sglang",
+                    version="0.5.16",
+                    path="/topk_method",
+                    expected="noaux_tc",
+                    reason="SGLang uses this field to register and route with the correction-bias parameter.",
+                    source=(
+                        "https://github.com/sgl-project/sglang/blob/v0.5.16/"
+                        "python/sglang/srt/models/deepseek_v2.py#L454-L475"
+                    ),
+                ),
+                HFConfigFieldDependency(
+                    engine="sglang",
+                    version="0.5.16",
+                    path="/moe_layer_freq",
+                    expected=1,
+                    reason="SGLang directly reads this field when deciding whether a decoder layer is sparse.",
+                    source=(
+                        "https://github.com/sgl-project/sglang/blob/v0.5.16/"
+                        "python/sglang/srt/models/deepseek_v2.py#L2190-L2196"
+                    ),
+                ),
+            ),
+        )
+
+        assert report.transformers_version == transformers.__version__
+        assert report.checked_engine_versions == ("vllm==0.26.0", "sglang==0.5.16")
+
     def test_from_hf_preserves_glm_specific_behavior(self):
         # 验证公共 HF 配置转换保留 GLM-5.2 的 DSA、router、MTP 及回写语义。
         hf_config = HFGlmMoeDsaConfig.from_pretrained(GLM5_2_TINY_MOE_PATH)

tests/utils/test_hf_config.py

@@ -0,0 +1,76 @@
+import json
+from pathlib import Path
+
+import pytest
+
+import transformers
+from transformers import AutoConfig, BertConfig
+from xtuner._testing import HFConfigFieldDependency, check_hf_config_save
+
+
+class _HFConfigExporter:
+    def __init__(self, source_hf_dir: Path, *, drop_field: str | None = None) -> None:
+        self.config = AutoConfig.from_pretrained(source_hf_dir)
+        self.drop_field = drop_field
+
+    def save_hf(self, output_dir: Path) -> None:
+        self.config.save_pretrained(output_dir)
+        if self.drop_field is not None:
+            config_path = output_dir / "config.json"
+            config = json.loads(config_path.read_text())
+            config.pop(self.drop_field)
+            config_path.write_text(json.dumps(config))
+
+
+def test_check_hf_config_save_uses_transformers_round_trip_as_reference(tmp_path: Path):
+    source_hf_dir = tmp_path / "source"
+    BertConfig(hidden_size=16, runtime_mode="special").save_pretrained(source_hf_dir)
+    source_config_path = source_hf_dir / "config.json"
+    source_config = json.loads(source_config_path.read_text())
+    source_config["transformers_version"] = "0.0.0"
+    source_config_path.write_text(json.dumps(source_config))
+
+    report = check_hf_config_save(
+        _HFConfigExporter(source_hf_dir),
+        source_hf_dir,
+        engine_dependencies=(
+            HFConfigFieldDependency(
+                engine="example-engine",
+                version="1.2.3",
+                path="/runtime_mode",
+                expected="special",
+                reason="The engine selects its runtime path from this field.",
+                source="https://example.com/example-engine/v1.2.3/model.py#L1",
+            ),
+        ),
+    )
+
+    assert report.transformers_version == transformers.__version__
+    assert report.transformers_normalized_fields == ("/transformers_version",)
+    assert report.checked_engine_versions == ("example-engine==1.2.3",)
+
+
+def test_check_hf_config_save_reports_dropped_fields(tmp_path: Path):
+    source_hf_dir = tmp_path / "source"
+    BertConfig(hidden_size=16).save_pretrained(source_hf_dir)
+
+    with pytest.raises(AssertionError) as error:
+        check_hf_config_save(
+            _HFConfigExporter(source_hf_dir, drop_field="hidden_size"),
+            source_hf_dir,
+            engine_dependencies=(
+                HFConfigFieldDependency(
+                    engine="example-engine",
+                    version="1.2.3",
+                    path="/hidden_size",
+                    expected=16,
+                    reason="The engine uses this field to construct parameter shapes.",
+                    source="https://example.com/example-engine/v1.2.3/model.py#L2",
+                ),
+            ),
+        )
+
+    message = str(error.value)
+    assert "Transformers direct round-trip" in message
+    assert "example-engine==1.2.3" in message
+    assert "/hidden_size" in message