Skip to content

dasLLAMA: size-tiered allocation alignment, tune-rail hardening, M1/zen2 re-baseline#3423

Open
borisbat wants to merge 19 commits into
masterfrom
bbatkin/dasllama-metal-baseline
Open

dasLLAMA: size-tiered allocation alignment, tune-rail hardening, M1/zen2 re-baseline#3423
borisbat wants to merge 19 commits into
masterfrom
bbatkin/dasllama-metal-baseline

Conversation

@borisbat

@borisbat borisbat commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

What

Four themes, one arc — the SPR-rematch fallout plus the M1/zen2 re-baseline it forced:

1. Size-tiered allocation alignment (das_aligned_alloc16)

platform.h: blocks ≥2KB now get cache-line (64B) alignment; blocks ≥2MB get 2MB alignment + MADV_HUGEPAGE on Linux (interior-only — hinting the partial tail would balloon RSS). The old uniform 16B left 75% of big SIMD panels line-split on every 512-bit/tile load (~260M split-loads/s measured on SPR AMX) and made per-launch heap luck flip kernel benches 4x bimodal. Windows stays uniform 64B (_aligned_msize must be called with the exact alloc-time alignment, so it cannot be size-tiered). Affects every das SIMD workload; zen2 gained +4-19% prefill from this alone.

2. Tune-rail hardening

  • dastest was tune-inert: aot_module = true on every test compile fed tune_aot_gate(), so [tune] kernels always compiled their reference bodies — the suite never exercised the tuned tier production runs use (82 vs 1600 GMAC/s on SPR fixtures).
  • tune_frozen policy — [tune] no longer piggybacks aot_module: aot_module means "module-shaped compile" (the -aot tool, dastest's test compiles, the AST serializer's module-symbol marking); borrowing it as the tune gate is what made dastest inert, and flipping it per-run broke AST serialization. New CodeOfPolicies.tune_frozen ("never apply per-box stamps") is set exactly where cross-box artifact soundness demands: the -aot driver and dastest's serialize path. tune_aot_gate() = tune_frozen || (aot && !jit_enabled) — an AOT-consuming run freezes tune only when the JIT is off (a stamp changes the semantic hash so a stamped function cannot AOT-link; under -jit the JIT supersedes AOT bodies and tuned stamps stay live).
  • AST serializer version gate: AstSerializer::getVersion() existed with no consumer — a stale stream misparsed field-by-field into an uncaught patch() throw (SIGABRT). Both entry points (Program::serialize, serializeProgram/module cache) now write/check the version and fail cleanly; deserialize_program catches dasException onto the failToCompile path. Serializer version 98 → 99 (tune_frozen in the CodeOfPolicies stream).
  • select_kernel_backend skipped the availability witness — for amx the witness doubles as the runtime enable (XTILEDATA arch_prctl), so explicit selection could SIGILL. It now runs the witness like pin and load-time auto-select already did.
  • gen_tune_probe benched every declined perm at best-of-N — 36 identical reference bodies measured separately (~4 min of duplicate work on arm64) that also handed the winner-picker 36 noise-lottery tickets. Declined perms bench once via the per-variant witness; M1 tune ~60s; the e2e confirm gate validated live (rejected a near-tied challenger 889 vs 895 tok/s).
  • The profiling drivers tune the box before measuring: gen_profile / gen_asr_profile carry [tune_policy(missing = "auto")], and the platform block records stamp provenance ("tune": "fname=suffix (manifest|fallback|reference)"). This gap cost a day: a deleted dasllama.tune.json silently put a whole M1 sweep on fallback stamps, ~5% low, with nothing in the output saying so.

3. M1 + zen2 re-baseline (new numbers, new methodology)

  • M1 (tuned, allocator fix, F16 KV): das wins prefill 16/16 (1.00-1.69x llama.cpp; E-series standouts E2B 1.68x / E4B 1.69x), trails emission 0.85-0.99x (memory-bound decode — the named next arc). Measured one-process-per-model: a 16-model single-process sweep understates late models (E2B measured 22% low from in-process heap accumulation; llama-bench gets a fresh process per model, now so do we).
  • zen2 (allocator fix, F16 KV; gen manifest absent — fallback stamps, so conservative): prefill +4-19% and emission flat-to-+12% vs the previous stored profile on all 16 models. ASR noise-equal. A tuned zen2 re-sweep is a follow-up when the box frees up.
  • F16-KV A/B on M1 (suspected regression): it is a win or wash at profile depths — Llama-3.2-1B prefill +10% under f16, Qwen3-4B tied.

4. Harness / rig additions

  • harness/backend_ops_shapes.patch: llama.cpp test-backend-ops never engages AMX (mmq needs weights in the AMX extra buft; the stock tool uses the default CPU buffer) — GGML_PERF_WEIGHT_BUFT in eval_perf fixes it (+51% verified). All prior stock-tool "lcpp AMX" per-op numbers anywhere were actually non-AMX.
  • harness/backend_probe.das: active-backend/witness diagnostic (which backend is live, is the gen family emitted) — earned its keep twice this arc.
  • benchmarks/matmul/ ladder rig: shape-matched das vs llama.cpp MUL_MAT bench (run_gemm_iso direct wrapper dodges the dastest tune-gate), --ngl Metal-offload baseline axis in the establisher.
  • GEMM generator: fused scale fold (llvm.fmuladd) + running-pointer amx fold addressing.
  • tests/dasLLAMA/test_batch_decode.das + test_prefill.das: the strict bit-exact/1e-6 batch-vs-sequential parity subtests assumed reference-tier kernels (the batch GEMM and the GEMV were literally the same dot loop). Under tuned kernels those tiers legitimately fold in different orders (a kstep4 tile vs a gkstep2 GEMV), so the strict q8 subtests now pin the portable backend (its batch and GEMV kernels share dot_q8q8 — same-dot parity holds under any stamp), and a new test_batch_decode_tuned subtest covers the active backend at token-exactness + a 1e-3 numeric budget. Both files were green on the untuned CI configuration before AND after; the pin makes them manifest-proof.

Verification

  • tests/dasLLAMA full suite: 321/321 under -jit at final tree state (tuned manifest present); also green with no manifest (the CI configuration).
  • --ser/--deser over the full tests/ tree: 10713/10713 both passes, plus corruption negatives — flipped version word → clean "stream version X does not match serializer version 99"; flipped structural length word → caught "ast serializer read overflow", clean per-program failure. No aborts.
  • Allocator behavior probed on both boxes (M1: 64B + 2MB tiers; zen2: 64B tier, by design).
  • Tune stamp provenance present in all four per-box profile JSONs (M1 written by the drivers; zen2 backfilled with the box's probe-verified fallback stamp — the box is busy, the tuned resweep is the documented follow-up).
  • Preflight --full: 17/17 gates green (format, lint, cpp-syntax full sweep, dasgen, ci-das, 6 doc gates + pdflatex, tests-cpp, interp, JIT, AOT, sequence smoke).

🤖 Generated with Claude Code

borisbat and others added 13 commits July 9, 2026 22:20
…e sessions

establish_baselines grows an --ngl flag (Metal-enabled clean build, separate
baseline_llm_<box>_metal.tsv via a suffix param on the baseline I/O) — first
M1 Max full-offload sweep committed: ~4.4x prefill / ~2.7-3.0x decode over the
best CPU rows at 4-8B. gen_profile sessions move to KVDtype.f16, matching the
create_session facade default and llama-bench's F16 KV reference config;
small-tier re-profile on master lands within noise (+-4.5%) as expected at
pp512/tg128 depths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…x fold addressing

The generator folds emitted bare FMul+FMul+FAdd — hand-built [llvm_code] IR never passes
das-level lowering, so mad()'s fusion never applied and no contract flags licensed the
backend to fuse: every family paid 3 FP ops per accumulator where lcpp's kernel pays 2.
fold_fma emits the same llvm.fmuladd das's mad() resolves to (fmul+fadd fallback when
unresolvable). Census: busd512 tile 24mul+12add -> 12mul+12fmadd; amx steady loop
128mul+64add -> 64mul+64fmadd. M1 kernel-iso: +2.9-3.9% on nrsplit2 tile rows, box winner
kstep4_nrsplit2_mr8_gkstep4 +2.0%.

amx fold addressing moves from precomputed pointer arrays (64 y rows + 32 xs rows, live
across the k-loop -> spilled, ~140 movq reloads per k-iteration = 28% of the steady loop)
to running pointers off three bases, lcpp's base + m*ldc form. Steady loop 494 -> 378
insns, movq 141 -> 89; now inside the lcpp mmq.cpp census window.

Gates: x64 emission 97/97, arm64 emission green, native grid GEN TUNE TEST OK,
tests/dasLLAMA 319/319 -jit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ench rig

The SPR-session opener: attributes prefill deltas to the GEMM kernels alone (no attention,
no KV, no model). bench_gemm_iso grows the fleet geometry table (smol135m/qwen06b/gemma1b/
llama1b/qwen4b/llama8b + the original tinyllama rows) selected via DASLLAMA_BENCH_MODEL,
NTOK override via DASLLAMA_BENCH_NTOK, reps from a fixed MAC budget. The lcpp half is
test-backend-ops perf -o MUL_MAT with the same shapes: harness/backend_ops_shapes.patch adds
the fleet cases + a GGML_BENCH_THREADS pin (upstream hardcodes hardware_concurrency).
harness/pure_gemm_ladder.sh runs both legs at one thread count and emits a merged TSV
(GFLOPS/TFLOPS -> GMAC/s normalized). gemm_1core_probe comment updated (examples/kernel-bench
no longer exists upstream; the ladder is the cross-engine instrument).

M1 shakeout @T8/512tok: lcpp CPU plateaus ~460-500 GMAC/s across fleet shapes; das arm64-gen
610-700. SPR runs the same instrument against the AMX build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
>=2KB blocks get cache-line (64B) alignment; >=2MB blocks get 2MB alignment +
MADV_HUGEPAGE on Linux; Windows stays uniform 64B (_aligned_msize must receive
the alloc-time alignment, so it cannot be size-tiered there).

The old uniform 16B alignment line-split every 512-bit vector load and AMX
tileloadd row on 75% of big-array allocations (~260M split-loads/s measured on
SPR AMX GEMM), and made per-launch allocation luck flip kernels between fast
and slow modes: das-amx qwen4b panels ran 4x bimodal across process launches.
Silicon-validated on Xeon 8488C: bimodality gone (12 runs, +-3% spread), every
shape at or above the old fast mode, cls +20%. THP A/B (enabled=never, same
alignment): the hugepage tier itself is +1-6% - the alignment tiers carry the
win. Intel SDM ch.20 mandates 64B for tile data.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- run_gemm_iso.das: direct-run wrapper. dastest compiles tests with
  policies.aot_module=true which trips tune_aot_gate() and stamps reference
  bodies for every [tune] kernel - the dastest rail silently benches the slow
  tier (82 vs 1600 GMAC/s on SPR). The M1 ladder rows in d553e5fa5 need
  re-measuring through this wrapper.
- bench_gemm_iso: pin_kernel_backend instead of select_kernel_backend for the
  repack half - pin runs the available() witness, which on amx stamps is the
  per-process XTILEDATA enable; bare select SIGILLs the first tile op.
- DASLLAMA_BENCH_SKIP_ROWMAJOR skips the row-major half for pin-lattice sweeps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… tail

Hinting the tail page backs it with a full 2MB of RSS (a 2.05MB block
balloons to ~4MB resident). Round the madvise length down to a 2MB multiple:
the streamed bulk keeps hugepages, the tail stays 4K, physical waste ~0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… rework

backend_ops_shapes.patch now also adds GGML_PERF_WEIGHT_BUFT to eval_perf:
quantized leaf tensors are placed into the matching CPU extra buffer type, so
test-backend-ops can actually measure llama.cpp's AMX mmq path (stock tool -
and upstream - never exercises it; weights sit in the default CPU buffer and
mmq requires the AMX buft). Verified on SPR: +51% on llama8b q at bs512, with
per-case engagement notes on stderr and a supports_op rollback for gated shapes.

pure_gemm_ladder.sh: das leg runs through run_gemm_iso.das (direct rail - the
dastest rail is tune-inert, see bench commit); lcpp leg becomes two rungs
(plain + lcpp-amx via the env; env stays UNSET for the plain rung - an empty
value strstr-matches any buft).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
aot_module=true on every test compile (both the run path in suite.das and the
serializer path) fed tune_aot_gate(), so [tune] kernels always compiled their
reference bodies — the suite never exercised the tuned tier production runs use.
C++-side the flag only affects module-shaped main files (ast_parse.cpp), which
test programs are not, so plain interp/jit test compiles are otherwise unchanged.

Known follow-up: tests/dasLLAMA/test_batch_decode.das q8 parity subtests assumed
reference-tier bit-parity between the batch GEMM and GEMV kernels and now fail at
tol 1e-6 under tuned kernels; fix lands separately.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
select_kernel_backend activated any registered backend without invoking
available() — for amx the witness doubles as the runtime enable (XTILEDATA
arch_prctl), so selecting it cold SIGILLed on the first tile op. Run the
witness like pin_kernel_backend and select_matmul_backend_for_load already
do; warn and keep the current backend when it reports unavailable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Declined perms all run the same reference body — timing each at best-of-ROUNDS
was pure duplicate measurement (a foreign-ISA box burned ~4 minutes measuring
one body 36 times) AND handed the winner-picker N noise-lottery tickets for a
single body. Keep one declined representative (the explicit 'reference' row when
present), erase the rest via the per-variant witness. M1 tune: ~4min -> ~60s;
the e2e confirm gate still guards a noise-crowned near-tie (validated live:
challenger 889 vs champ 895 tok/s -> rejected, champ kept).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…venance

gen_profile / gen_asr_profile carry [tune_policy(missing = "auto")] (the
server's zero-config rail): an untuned box runs the scope tuner and re-execs
with winners stamped, and log_tune_status reports the stamp source at startup.
The platform block records provenance ('tune': 'fname=suffix (manifest|
fallback|reference)') via gather_platform, so fallback-tier numbers can never
masquerade as tuned ones — a deleted dasllama.tune.json silently cost ~5%
prefill across a whole published M1 sweep before this.

harness/backend_probe.das: tiny active-backend/witness diagnostic (which
backend is live post-load, is the gen family emitted).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…smoke

The q8 bit-exact/1e-6 batch-vs-sequential contracts in test_batch_decode and
test_prefill held only while the batch tile and the GEMV compiled the same fold
order — true for reference bodies and for the per-ISA fallback stamps, broken
by design once a tune manifest stamps e.g. a kstep4 tile next to a gkstep2
GEMV. The strict subtests now pin the portable backend (its batch and GEMV
kernels share dot_q8q8, so same-dot parity holds under any stamp) and clear the
pin on exit; new test_batch_decode_tuned covers the active backend at greedy-
token exactness plus a 1e-3 numeric budget. Green with and without a manifest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
M1 (tuned manifest, allocator fix, F16 KV, one process per model — an in-process
16-model sweep understates late models via heap accumulation; E2B measured 22%
low): das wins prefill 16/16 vs llama.cpp (1.00-1.69x; E2B 1.68x / E4B 1.69x),
trails emission 0.85-0.99x (memory-bound decode, the named next arc). Platform
blocks now carry the tune stamp provenance.

zen2 (allocator fix, F16 KV, 16 threads; gen manifest absent -> fallback maddubs
stamps, i.e. conservative): prefill +4-19% and emission flat-to-+12% vs the
prior stored profile on all 16 models; ASR noise-equal. Tuned zen2 resweep is a
follow-up when the box frees up.

Scoreboards regenerated from the JSONs by gen_results.das.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 10, 2026 05:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens dasLLAMA’s tuning/profiling pipeline and updates performance methodology/results, alongside a core allocator alignment change intended to improve large-SIMD/GEMM behavior across platforms.

Changes:

  • Introduces size-tiered allocation alignment in das_aligned_alloc16 (64B for ≥2KB; 2MB + Linux hugepage hint for ≥2MB; Windows fixed 64B).
  • Fixes tune-rail correctness (AOT gating in dastest, backend explicit selection now runs availability witness) and adds tune provenance logging into profile JSON.
  • Updates/extends dasLLAMA tests and harnesses (portable pin for strict parity tests, adds a tuned-tolerance batch decode test, adds pure-GEMM ladder tooling and shape patch for llama.cpp).

Reviewed changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/dasLLAMA/test_prefill.das Pins portable backend for strict numeric parity in prefill test; clears pin afterward.
tests/dasLLAMA/test_batch_decode.das Pins portable backend for strict parity subtests; adds tuned-tolerance batch test.
modules/dasLLAMA/tune_for_this_box.md Documents new tune/profiling behavior and provenance.
modules/dasLLAMA/performance/results_llm.md Regenerated LLM results tables with new measurements/metadata.
modules/dasLLAMA/performance/profile_llm_zen2.json Updated zen2 LLM profile numbers + metadata.
modules/dasLLAMA/performance/profile_llm_m1.json Updated M1 LLM profile numbers + adds tune provenance field.
modules/dasLLAMA/performance/profile_common.das Adds optional tune field and baseline path suffix support.
modules/dasLLAMA/performance/profile_asr_zen2.json Updated zen2 ASR profile numbers + metadata.
modules/dasLLAMA/performance/profile_asr_m1.json Updated M1 ASR profile numbers + adds tune provenance field and librispeech stats.
modules/dasLLAMA/performance/gen_profile.das Adds [tune_policy], logs tune status, and uses F16 KV for measurement sessions.
modules/dasLLAMA/performance/gen_asr_profile.das Adds [tune_policy] and logs tune status before profiling.
modules/dasLLAMA/performance/establish_baselines.das Adds --ngl axis and writes per-axis baseline TSVs (e.g. _metal).
modules/dasLLAMA/performance/baseline_llm_m1_metal.tsv New baseline TSV for Metal offload axis.
modules/dasLLAMA/harness/pure_gemm_ladder.sh New script to compare das vs llama.cpp GEMM throughput on matched shapes.
modules/dasLLAMA/harness/gen_tune_probe.das Dedupes declined-permutation benchmarking to avoid redundant work/noise bias.
modules/dasLLAMA/harness/gemm_1core_probe.das Updates harness commentary to align with new ladder tooling.
modules/dasLLAMA/harness/backend_probe.das New diagnostic to print active backend/witness status around a model load.
modules/dasLLAMA/harness/backend_ops_shapes.patch Patch for llama.cpp test-backend-ops to add shapes + buffer controls for ladder parity.
modules/dasLLAMA/dasllama/dasllama_math.das select_kernel_backend now runs availability witness (avoids unsafe activation).
modules/dasLLAMA/dasllama/dasllama_gemm_gen.das Adds fmuladd folding and AMX fold addressing tweaks for generator output.
modules/dasLLAMA/benchmarks/matmul/run_gemm_iso.das New direct-run wrapper to avoid tune-gate issues in dastest.
modules/dasLLAMA/benchmarks/matmul/bench_gemm_iso.das Extends bench to multiple model shape sets and NTOK override; uses repack tier pin.
modules/dasLLAMA/benchmarks/asr/results.md Regenerated ASR results tables.
include/daScript/misc/platform.h Implements size-tiered alignment + Linux hugepage hint; Windows alignment updated.
dastest/suite.das Fixes aot_module to track ctx.use_aot (tune no longer inert under tests).
dastest/dastest.das Same aot_module fix for serialization path.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread modules/dasLLAMA/dasllama/dasllama_gemm_gen.das
Comment thread modules/dasLLAMA/benchmarks/matmul/bench_gemm_iso.das
Comment thread modules/dasLLAMA/harness/pure_gemm_ladder.sh Outdated
borisbat and others added 2 commits July 10, 2026 01:26
bench_gemm_iso clears its backend pin on exit — with dastest no longer
tune-inert, dastest-driven bench runs are meaningful and a standing global pin
would leak into later files in the same process. pure_gemm_ladder.sh gets
pipefail so a daslang failure can't hide behind a successful awk at the end of
a pipe (a measurement harness must not emit partial TSV silently).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…module

aot_module means 'module-shaped compile' (the -aot tool, dastest's every test
compile, the AST serializer's module-symbol marking) — borrowing it as the tune
gate made dastest tune-inert, and flipping it per-run broke AST serialization
(AstSerializer::patch() read past end of stream on --deser; extended_checks red
on all three OSes).

New CodeOfPolicies.tune_frozen ('never apply per-box [tune] stamps'), set
exactly where cross-box artifact soundness demands it: the -aot driver and
dastest's serialize path. tune_aot_gate() becomes
tune_frozen || (aot && !jit_enabled): an AOT-consuming run freezes tune only
when the JIT is off — a stamp changes the semantic hash so a stamped function
cannot AOT-link, but under -jit the JIT supersedes AOT bodies and tuned stamps
stay live. dastest's run path returns to its original aot_module = true.

AST serializer version 98 -> 99 (CodeOfPolicies stream grows the field).
Verified: --ser/--deser over tests/ 10713/10713 both passes; tests/dasLLAMA
321/321 under -jit with stamps live (the tune line now appears under plain
dastest — the original goal, without the borrowed flag).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 10, 2026 09:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated 2 comments.

Comment thread src/builtin/module_builtin_ast_serialize.cpp
Comment thread modules/dasLLAMA/dasllama/dasllama_gemm_gen.das
…rialize

AstSerializer::getVersion() existed as a bump-discipline constant with no
consumer — a stale stream misparsed field-by-field until a patch() throw
escaped rtti_ast_serializer_deserialize_program uncaught (the SIGABRT
extended_checks showed on all three OSes this PR). Program::serialize now
writes/checks the version at the head of every program record (a desynced
stream keeps failing cleanly on each record), and deserialize_program wraps
the read in the same dasException catch serializeScript already uses, so
same-version truncation/corruption lands on the failToCompile path too.

Verified: --ser/--deser over tests/ 10713/10713; corrupted version word ->
clean 'stream version X does not match serializer version 99'; corrupted
structural length word -> caught 'ast serializer read overflow', clean
per-program failure report. No aborts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 10, 2026 09:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated 1 comment.

Comment thread src/builtin/module_builtin_ast_serialize.cpp
…rogram)

The cache path (trySerializeProgramModule) checks only mtime+filename before
handing the stream to serializeProgram — a cache written by a different
serializer version misparsed from the first layout difference. Same gate as
Program::serialize; on mismatch ser.failed is set and the caller's existing
failed-check falls back to a full parse.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 10, 2026 09:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated 2 comments.

Comment thread modules/dasLLAMA/harness/pure_gemm_ladder.sh
Comment thread modules/dasLLAMA/harness/backend_ops_shapes.patch Outdated
… clamps thread env

env -u GGML_PERF_WEIGHT_BUFT on the plain rung — an exported var would silently
turn the baseline into the AMX rung (and empty is NOT unset: an empty value
strstr-matches every buft). GGML_BENCH_THREADS in backend_ops_shapes.patch now
falls back to hardware_concurrency on unset/zero/garbage instead of casting
atoi straight to size_t. Patch re-verified to apply clean on upstream ebd048f.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 10, 2026 10:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated 2 comments.

Comment thread modules/dasLLAMA/performance/profile_llm_zen2.json
Comment thread modules/dasLLAMA/performance/profile_asr_zen2.json
…obe-verified)

The zen2 profiles were generated before the platform.tune field existed; the
value is backfilled from the box's own backend_probe output (x64-gen active,
no manifest -> 'q8q8_tile_gen=dot_maddubs_width256_mr8_kstep2 (fallback)'),
measurement data untouched. Makes the fallback-tier caveat auditable in the
artifact itself; the tuned zen2 resweep stays the documented follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 10, 2026 10:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated 1 comment.

Comment thread modules/dasLLAMA/harness/gen_tune_probe.das

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

modules/dasLLAMA/dasllama/dasllama_math.das:986

  • pin_kernel_backend already invokes the backend’s available witness before activation, and select_kernel_backend now invokes the same witness as well. Because pin_kernel_backend ends by calling select_kernel_backend(name), pinning a backend will run the witness twice (which is redundant and can be undesirable if the witness has side effects like AMX enablement). Consider activating directly after the successful witness check in pin_kernel_backend to avoid the double-invoke.
    activate(g_kernel_backends[name])
}

//! Pin the active backend to `name` (forces it; suppresses auto-select on later registrations). For A/B
//! benchmarking on one binary — e.g. pin "arm64-sdot" to measure the row-major baseline against the gen tier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants