perf(advisor): recommend Turbo KV-cache modes per family and context

[inureyes]

Summary

Extend the existing quant/cache advisor so mlxcel generate --recommend-quant also emits an advisory KV-cache-mode suggestion per model family and context range, among fp16, int8, fp16+turbo4 (asymmetric Turbo4), turbo4 (symmetric), and fp16+turbo3. This is a P3 backlog item: the goal is to give operators a benchmark starting point for long-context and memory-constrained serving, not to change any default.

The recommendations are advisory and opt-in only. The default inference path is unchanged: with no flags resolve_kv_cache_mode still returns fp16, and nothing is auto-applied. The advisor reads only config.json, prints suggestions, and exits.

What changed

• New module src/execution/kv_cache_advisor.rs: the pure recommendation core recommend_kv_cache_mode(arch_kind, model_type, range), the config-driven advise_kv_cache_modes(model_path), the KvContextRange bucket (short <=4K, medium 4K-32K, long >32K), the KvCacheModeAdvice value, and render_kv_cache_advice / print_kv_cache_advice.
• It keys on the KV-architecture class already computed by src/execution/kv_arch.rs (standard, sliding-window, MLA, hybrid, pure SSM) plus the raw model_type for the symmetric-Turbo4 PPL allowlist (src/lib/mlxcel-core/src/cache/turbo/allowlist.rs).
• QuantAdvice gains a kv_cache_advice: Vec<KvCacheModeAdvice> field, populated in advise_quantization and rendered by print_quant_advice, so --recommend-quant surfaces the suggestions with no behavior change to any other caller.
• docs/turbo-kv-cache.md documents the new --recommend-quant surface: the family/context keying, the conservative rule table, the advisory/opt-in framing, and the no-weight-promotion guarantee.

No bf16-to-f16 quantized-weight promotion (the #289 landmine)

A recommendation is always one of the five KVCacheMode values. KV-cache modes quantize only the K/V cache tensors and dequantize back to FP16 for SDPA; they never touch model weights or their scales/biases. The advisor never calls into the weight-loading / sanitize.rs path, so it cannot reintroduce the bf16-to-f16 quantized-weight promotion that caused the #289 decode regression. Symmetric turbo4 is suggested only for families on the PPL allowlist, and MLA / pure-SSM families never receive a Walsh-Hadamard Turbo mode (their cache dimension is not a power of two, or absent). A unit test enumerates every (architecture, family, range) combination and asserts the output stays within the five allowed modes and never turbo4-delegated.

Test plan

• cargo check -p mlxcel --lib --release --features metal,accelerate
• cargo test --release -p mlxcel --lib --features metal,accelerate kv_cache_advisor:: (15 tests: range bucketing, short-context-is-fp16, symmetric-turbo4-only-for-allowlisted, MLA-uses-int8-never-turbo, pure-SSM-fp16, always-an-allowed-mode, default-unchanged-without-opt-in, advisory-framing render)
• cargo clippy -p mlxcel --lib --tests --release --features metal,accelerate clean for the changed files
• cargo fmt --check clean for the changed files
• python3 scripts/ci/check_cross_repo_refs.py

Remaining acceptance-criteria items (recorded 4K/16K/32K sweeps across two dense and two MoE/VLM families, and the recorded per-family quality guard) are intentionally left open: they are the validation coverage that keeps the recommendations advisory. The orchestrator runs the real-model KV-mode smoke after merge.

Closes #327

[inureyes]

PR Finalization

Tests

19 advisor unit tests pass. No gaps found; the existing suite covers context-range bucketing, per-family recommendations, allowlist gating, non-power-of-two head-dim downgrade, default-unchanged invariant, render output, and the alternate field-name guard added in 0b24f0f.

Documentation

docs/turbo-kv-cache.md had one stale sentence: the head-dimension derivation description mentioned only hidden_size / num_attention_heads, but 0b24f0f broadened the field-name coverage to also handle d_model / dim / model_dim and num_heads / n_heads / n_head. Updated to list the full set and note that it mirrors the kv_arch classifier. All four advisory requirements are present and accurate: the section describes --recommend-quant, marks the output as advisory/opt-in, states the default is unchanged, notes KV modes never touch model weights (no bf16->f16 weight promotion), and covers the non-power-of-two head-dim fallback.

No Korean mirror of this doc exists in the repo.

Lint / Format

cargo fmt --check clean. cargo clippy -p mlxcel --lib --tests --features metal,accelerate -- -D warnings exits 0 with no diagnostics.

Commit: d4a0445

[inureyes]

Real-model validation (orchestrator)

--recommend-quant smoke on M1 Ultra:

• qwen2.5-7b-4bit (non-allowlisted dense): per-range KV advice prints. short = fp16; medium/long = fp16+turbo4 (also int8 / fp16+turbo3). Symmetric turbo4 is correctly withheld because the family is not on the PPL allowlist.
• qwen3.5-4b-4bit (allowlisted): long = turbo4 (symmetric, ~73% KV savings), correctly differentiated by is_symmetric_turbo_allowed; fp16+turbo4 offered as the lower-risk fallback.
• The output states the advice is advisory/opt-in and that the default inference path is unchanged (fp16).
• Recommended modes load and generate coherently: qwen2.5-7b fp16+turbo4 OK, qwen3.5-4b turbo4 OK.
• Default (no kv flag) unchanged: qwen2.5-7b runs fp16, coherent, 85 tok/s.

Reviewer and security-checker each fixed a MEDIUM head_dim guard (a non-power-of-2 head_dim family could otherwise be recommended a Turbo mode that panics on opt-in; now downgraded to int8/fp16, with the field-name set broadened to match the arch classifier). Finalizer corrected a stale doc sentence. 0 unresolved CRITICAL/HIGH. Acceptance criteria met (advisory/opt-in, no bf16->f16 quant-weight promotion, sound per-family/context recommendations, integrated into --recommend-quant).