Skip to content

Add CogVideoX Diffusers checkpoint conversion - #14651

Open
anubhavsingh23906-code wants to merge 1 commit into
huggingface:mainfrom
anubhavsingh23906-code:add-cogvideox-reverse-conversion
Open

Add CogVideoX Diffusers checkpoint conversion#14651
anubhavsingh23906-code wants to merge 1 commit into
huggingface:mainfrom
anubhavsingh23906-code:add-cogvideox-reverse-conversion

Conversation

@anubhavsingh23906-code

Copy link
Copy Markdown

What does this PR do?

Adds a new conversion script to convert CogVideoX transformer checkpoints from Diffusers format to the CogVideoX checkpoint format.

Changes

  • Added scripts/convert_diffusers_to_cogvideox.py
  • Converts separate Q/K/V projection weights into combined QKV weights.
  • Converts query/key LayerNorm keys to the CogVideoX format.
  • Reconstructs AdaLN modulation weights and biases.
  • Converts transformer layer key names to the expected CogVideoX format.
  • Added CLI arguments for input and output checkpoint paths.

Validation

  • Verified QKV reconstruction using synthetic tensors.
  • Verified AdaLN weight rearrangement.
  • Verified query/key LayerNorm key conversion.
  • Verified CogVideoXTransformer3DModel can be instantiated successfully.
  • Verified CLI argument parsing with --help.
  • git diff --check passes.

Testing

The conversion logic has currently been validated with synthetic state dictionaries. Real checkpoint conversion has not yet been performed because a suitable source checkpoint was not available locally.

Dependencies

No new dependencies were added.

Self-review

  • Reviewed the conversion logic for QKV, AdaLN, LayerNorm, and transformer key mappings.
  • Confirmed the script passes basic syntax/CLI and synthetic conversion checks.
  • Real checkpoint conversion remains to be validated separately.

@github-actions github-actions Bot added the size/L PR with diff > 200 LOC label Aug 30, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Hi @anubhavsingh23906-code, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked. See the contribution guide for more details. If this PR intentionally does not fix a tracked issue, a maintainer can add the no-issue-needed label to silence this reminder.

Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice.

Once the PR links an issue (or gets the no-issue-needed label), you can ignore this message — it stays here as a comment, but it no longer applies.

@anubhavsingh23906-code

Copy link
Copy Markdown
Author

Thanks for the reminder.

I checked the existing Diffusers issues and could not find a tracked issue specifically
requesting the reverse CogVideoX checkpoint conversion implemented by this PR.

This PR is intended to add the missing reverse conversion utility rather than fix a
specific bug. Could a maintainer please add the no-issue-needed label if appropriate,
or point me to an existing issue/discussion that this work should be linked to?

Hi @anubhavsingh23906-code, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked. See the contribution guide for more details. If this PR intentionally does not fix a tracked issue, a maintainer can add the no-issue-needed label to silence this reminder.

Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice.

Once the PR links an issue (or gets the no-issue-needed label), you can ignore this message — it stays here as a comment, but it no longer applies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L PR with diff > 200 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant