Skip to content

benchmarks: add Apple I8MM affine-QMM evidence harness - #4235

Draft
PhilipJohnBasile wants to merge 1 commit into
ml-explore:mainfrom
PhilipJohnBasile:agent/mlx-highway-i8mm-20260813
Draft

benchmarks: add Apple I8MM affine-QMM evidence harness#4235
PhilipJohnBasile wants to merge 1 commit into
ml-explore:mainfrom
PhilipJohnBasile:agent/mlx-highway-i8mm-20260813

Conversation

@PhilipJohnBasile

Copy link
Copy Markdown
Contributor

Summary

This draft adds an opt-in Apple arm64 evidence harness for comparing MLX affine 8-bit QMM against two dynamic-int8 activation controls:

  • a U8×I8 NEON dot-product implementation
  • a 2×2 U8×I8 I8MM implementation using vusmmlaq_s32

It follows the closed #4197 discussion and is intended only to inform a possible future ARM path alongside Highway work in #3019.

Scope

This draft:

  • does not alter MLX production dispatch
  • is excluded from normal builds unless explicitly enabled
  • records activation preparation separately from same-preparation compute
  • separates numerical-correctness runs from timing-only runs
  • embeds the candidate source SHA256 and rejects a mismatched candidate
  • records MLX binary/source identities as unbound provenance
  • permanently reports throughput: null

It is not evidence of a public speedup, model-level quality, production readiness, Highway integration, or MLX source/binary provenance.

Why this shape

The earlier microbenchmark compared against a scalar stand-in and mixed activation-reuse gains with I8MM gains. This harness adds an actual MLX baseline, a same-preparation dot-product control, alternating measurement order, tail/edge coverage, and explicit fail-closed claim boundaries.

Validation

  • fresh opt-in CMake configure and target build
  • source-identity check and mismatch rejection
  • normal self-test passed
  • ASan and UBSan self-test passed
  • diagnostic M=1/2/3 comparison passed
  • CMake, C++, Python, and diff formatting checks passed

Possible extraction path

If the evidence eventually supports production work, activation reuse, the I8MM tile, and ARM Highway/CMake dispatch should be developed and reviewed as separate changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RFC: enable NEON I8MM for CPU quantized matmul (8-bit) — 7-24x measured on M5

1 participant