feat(ds4): add fused HIP decode for ROCmFPX

[davide221]

Summary

Add the production DeepSeek V4 Flash decode path on top of the ROCmFPX types
merged in #502:

• cached, monolithic HIP decode graph covering HC, attention, MoE, and logits;
• gfx1151 single-column ROCmFP2/FP3 fixed-K kernels and ROCmFP4 loop
  specialization, restricted to the exact profiled shapes;
• explicit --ds4-fused-decode and --ds4-expert-top-k N serving options;
• exact tokenwise prefill and cache-reset/ring-wrap correctness fixes;
• race-free persistent HC worker pool for the layered/hybrid fallback.

The branch is rebuilt directly on current main; it no longer carries the
already-merged #502 changes or the old submodule-era conflicts.

Runtime behavior

--ds4-fused-decode now requests a monolithic HIP model load because the fused
graph must reference every expert tensor directly. If that allocation fails,
the backend logs the fallback and disables fused execution for hybrid expert
placement instead of claiming that the fused graph is active.

--ds4-expert-top-k 4 keeps and renormalizes four routed experts. It is an
explicit approximate-inference policy; 0 retains the model's configured
expert count.

No performance feature in this PR requires a serving-time environment
variable. DFLASH_DS4_TIMING remains optional instrumentation only.

Correctness and maintainability

• New sequences reset both cache storage and compressed-row topology.
• Decode after raw-SWA ring wrap reads the correct physical rows; unit coverage
  exercises pre-wrap, boundary, and wrapped cases.
• Compressed-KV writes are graph dependencies, and cached graph keys include
  the flush topology.
• Fused timing is included in phase-level telemetry.
• Default MoE reduction keeps the proven permute + cont + sum_rows path.
• No implicit system prompt or unsupported chunked-prefill mode is introduced.
• Dead full-step graph-cache code was removed.
• The HC worker sequence and stop predicates are synchronized with the same
  mutex used by the condition-variable wait, preventing lost wakeups.

Validation

Hardware and model:

• Strix Halo gfx1151, ROCm 7.2.2, 128 GiB RAM / 96 GiB ROCm aperture;
• performance profile with 2900 MHz SCLK observed throughout the run;
• DeepSeek-V4-Flash-ROCMFP2-STRIX.gguf, SHA-256
  8fa6c30d9badd8e72f83c62952fe24b2e8cd647e103aff086bece328cb9c9208.

Configuration:

--target-device hip:0
--max-ctx 8192
--prefix-cache-slots 0
--prefill-cache-slots 0
--ds4-fused-decode
--ds4-expert-top-k 4

Two consecutive GSM+Math runs produced identical generated text and scores:

Suite	Accuracy per run	Mean decode	Output tokens
GSM	5/5	26.39 tok/s	632
Math	5/5	25.83 tok/s	963
Combined	10/10	26.11 tok/s unweighted; 25.89 token-weighted	1,595

The same checked prompt measured 16.10 tok/s when the old 8K initialization
silently selected hybrid/layered execution and 26.56 tok/s after the monolithic
fused path was made reachable.

The strict trace gate compared 64 full-vocabulary F32 logits rows byte for byte
with zero mismatches. The final model-loading fix changes path selection only;
it does not change graph arithmetic. A fresh ROCm build passes all 20 CTest
targets, including ROCmFP reference/tail tests, RMSNorm, HC, DeepSeek4, and
server tests.

Top-k 4 still requires workload-level quality validation beyond this small
regression suite.

[cubic-dev-ai]

All reported issues were addressed across 6 files

Not reviewed (too large): server/src/deepseek4/deepseek4_graph.cpp (~1,745 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[davide221]

Validation update (2026-07-09): validated on lucebox2 against DeepSeek-V4-Flash ROCmFP2. The env-only expert routing knob DFLASH_DS4_TOPK=4 reduces routed experts from 6 to 4 and renormalizes retained weights. AR validation (GSM n=5 plus Math n=5) reached 26.19 tok/s GSM and 25.16 tok/s Math, both 5/5, versus baseline 23.54 and 22.24 tok/s. DSpark fused speculative decode reached 20.1–20.3 tok/s warm in repeated q=4 runs, with approximately 105–106 ms verify steps. Adaptive width, causal batched verify, fused verify, and interior-boundary compressor emission are wired and producing coherent output. The neuron-magnitude mask preserved the small-screen scores but produced no speedup; the block-sparse follow-up was not quality-safe and is kept out of the validated path. These measurements come from the hub-494 experiment worktree; the final k=4 patch and larger DSpark runtime follow-on still require explicit commit/push if they are to become part of the PR diff.

[cubic-dev-ai]

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/src/deepseek4/deepseek4_graph.cpp">

<violation number="1">
P2: `DFLASH_DS4_TOPK` will not reduce expert work for hash-routed DS4 layers: this override only changes `build_moe_routing`, while the hash-routed decode graphs still allocate/upload `w.n_expert_used` expert IDs and run `ggml_mul_mat_id` over that full count. If the knob is intended to be a serving-wide bandwidth lever, it would be worth applying the same reduced count when building and filling `hash_ids`, or documenting that hash-routed layers are excluded.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[DeanoC]

Semi-Automated review pass.
Codex: Overall this is careful, well-documented work and the risky paths (fused decode, FFN combine variants, GPU HC) are correctly gated behind default-OFF env flags. My comments concentrate on a few changes that land on the default code path plus one likely-dead block. Inline notes below;

Added a few from Codex, all just nits really. Works well, i've been tested it over the last few days and is works well!
Nice work davide!

[cubic-dev-ai]

No issues found across 8 files

Not reviewed (too large): server/src/deepseek4/deepseek4_graph.cpp (~1,795 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.

_{Re-trigger cubic}