fix(pt): HybridMuon ZeRO checkpoint slowdown#5525
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe PR refactors parameter naming registration in ChangesHybridMuon Parameter Naming Refactor
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #5525 +/- ##
==========================================
- Coverage 82.19% 82.19% -0.01%
==========================================
Files 891 891
Lines 101599 101605 +6
Branches 4242 4240 -2
==========================================
+ Hits 83507 83510 +3
- Misses 16789 16791 +2
- Partials 1303 1304 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Summary
param_groups.ZeroRedundancyOptimizer.consolidate_state_dict()from gathering a duplicate model-sizednamed_parametersobject graph.Test plan
HybridMuon.optimizer.param_groups[0]["named_parameters"].Summary by CodeRabbit