Skip to content

feat: load scaled FP8 weights without upfront conversion - #1913

Open
leejet wants to merge 1 commit into
masterfrom
feat/native-fp8-weight-loading
Open

feat: load scaled FP8 weights without upfront conversion#1913
leejet wants to merge 1 commit into
masterfrom
feat/native-fp8-weight-loading

Conversation

@leejet

@leejet leejet commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Summary

  • preserve E4M3 and E5M2 safetensors weights in their original one-byte representation
  • cast active FP8 linear weights to BF16 at runtime and apply scalar or per-channel weight scales
  • normalize .scale_weight tensor names to .weight_scale
  • align sd_type_t with the new ggml FP8 types and update the ggml submodule

Related Issue / Discussion

N/A

Additional Information

N/A

Checklist

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant