Skip to content

vulkan: register PTQ1_0 in the selection and supports_op enumerations - #149

Closed
bri-prism wants to merge 1 commit into
feat/tq1_0-g128from
fix/ptq1_0-vulkan-supports-op
Closed

vulkan: register PTQ1_0 in the selection and supports_op enumerations#149
bri-prism wants to merge 1 commit into
feat/tq1_0-g128from
fix/ptq1_0-vulkan-supports-op

Conversation

@bri-prism

Copy link
Copy Markdown

Stacked on #148. Without this, PTQ1_0's Vulkan backend builds green and executes nothing.

The problem

Every PTQ1_0 reference in ggml-vulkan.cpp sits in pipeline creation (lines 4637–5466). None are in the selection or support enumerations. So ggml_backend_vk_device_supports_op returns false, the scheduler never offers MUL_MAT to Vulkan, and the 54 generated SPIR-V shaders are unreachable code.

This is the "registers but is never selected" failure mode, and it is worse than a visible failure because every signal reads healthy — clean build, shaders generated, test-backend-ops reporting 1009/1009 ... Backend Vulkan0: OK. On inspection all 78 ptq1_0 lines said not supported [Vulkan0].

The fix

case GGML_TYPE_PTQ1_0: alongside the existing GGML_TYPE_Q1_0 in the seven enumerations that have working PTQ1_0 pipelines behind them:

function role
ggml_vk_get_to_fp16 dequant-to-fp16 selection
ggml_vk_get_mul_mat_mat_pipeline matmul selection
ggml_vk_get_dequantize_mul_mat_vec mat-vec selection
ggml_vk_get_mul_mat_mat_id_pipeline MoE matmul selection
ggml_vk_get_dequantize_mul_mat_vec_id MoE mat-vec selection
ggml_backend_vk_device_supports_op MUL_MAT / MUL_MAT_ID gate
ggml_backend_vk_device_supports_op GET_ROWS gate

Five sites deliberately left alone

There are twelve case GGML_TYPE_Q1_0: in this file, not seven. Do not blanket-add:

  • ggml_vk_get_cpy_pipeline (×2)
  • supports_opGGML_OP_SET_ROWS
  • supports_opGGML_OP_DUP (×2)

All five depend on copy_to_quant.comp, which has no PTQ1_0 entry — quantizing to PTQ1_0 on device is unimplemented. Adding them would advertise a path whose shader does not exist, which is the inverse of the coopmat2 break fixed in #148 (there, a shader generated with no decoder; here, a path advertised with no shader). A clean refusal is correct until copy_to_quant gains a PTQ1_0 entry.

Measured

Intel Arc B390, KHR_coopmat, proprietary Windows driver. test-backend-ops test -b Vulkan0 -o MUL_MAT, two runs, ptq1_0 per-case output byte-identical:

type OK not supported FAIL
q1_0 28 50 0
q2_0 28 50 0
ptq1_0 28 50 0
tq2_0 11 0 0

1037/1037 tests passed, 0 failures. Before this patch: ptq1_0 was 0 OK / 78 not supported.

Two checks worth stating explicitly, because a green run proved nothing here twice already:

  • OK counts, not line counts. grep -c type_a=ptq1_0 counts the skip lines too; that is how the pre-patch state read as a pass.
  • Total case count rose 1009 → 1037, which is independent evidence that work was added rather than reshuffled.

28 is the full available set, not a partial pass — the 50 declines are shape and permutation variants that q1_0 and q2_0 also decline on this device.

The PTQ1_0 kernel itself needed no changes. It was correct; it was unreachable. Mirroring Q1_0 rather than Q2_0 was the right call.

PTQ1_0's Vulkan pipelines are created but nothing can select them. Every
PTQ1_0 reference in ggml-vulkan.cpp sits in pipeline creation (4637-5466)
and none in the selection or support enumerations, so
ggml_backend_vk_device_supports_op returns false, the scheduler never
offers MUL_MAT to Vulkan, and the 54 generated SPIR-V shaders are
unreachable. The build is green and the backend does nothing.

Add `case GGML_TYPE_PTQ1_0:` alongside the existing GGML_TYPE_Q1_0 in the
seven enumerations that have working PTQ1_0 pipelines behind them:

  ggml_vk_get_to_fp16
  ggml_vk_get_mul_mat_mat_pipeline
  ggml_vk_get_dequantize_mul_mat_vec
  ggml_vk_get_mul_mat_mat_id_pipeline
  ggml_vk_get_dequantize_mul_mat_vec_id
  ggml_backend_vk_device_supports_op   (MUL_MAT / MUL_MAT_ID gate)
  ggml_backend_vk_device_supports_op   (GET_ROWS gate)

There are twelve `case GGML_TYPE_Q1_0:` in this file; five are deliberately
left alone. ggml_vk_get_cpy_pipeline (x2), GGML_OP_SET_ROWS and GGML_OP_DUP
(x2) all depend on copy_to_quant.comp, which has no PTQ1_0 entry, so
quantizing to PTQ1_0 on device is unimplemented. Advertising those paths
would claim support for a shader that does not exist; a clean refusal is
correct until copy_to_quant gains a PTQ1_0 entry.

Measured on Intel Arc B390 (KHR_coopmat, proprietary Windows driver),
test-backend-ops test -b Vulkan0 -o MUL_MAT, two runs, byte-identical:

  type      OK   not_supported  FAIL
  q1_0      28   50             0
  q2_0      28   50             0
  ptq1_0    28   50             0     (was 0 OK / 78 not_supported)
  tq2_0     11    0             0

  1037/1037 tests passed, 0 failures; total case count rises 1009 -> 1037.

28 is the full available set rather than a partial pass: the 50 declines are
shape and permutation variants that q1_0 and q2_0 also decline on this
device. The PTQ1_0 kernel itself needed no changes - it was correct, just
unreachable.
@bri-prism

Copy link
Copy Markdown
Author

Superseded by 705d9cc on feat/tq1_0-g128 — these seven lines are now in the base, with a necessary guard I had missed.

My patch was incomplete and would have crashed on coopmat2 hardware. Registering PTQ1_0 in supports_op makes the op reachable, but PTQ1_0 is the only type in those lists with no coopmat2 pipelines (deliberately — dequant_funcs_cm2.glsl has no decoder, so vulkan-shaders-gen skips cm2 for it). I assumed the caller's mmp == nullptr fallback would absorb that. It does not: vk_matmul_pipeline2() make_shareds both accumulators, so an unpopulated cm2 slot is a valid pointer whose l/m/s are all null. The mm getter would return it and guess_matmul_pipeline would hand back a null pipeline; the mul_mat_id getter would reach GGML_ASSERT(support_fp32acc) and abort.

The fix returns nullptr when the cm2 slots are empty, routing to the dequant + f16 matmul fallback the callers already implement — the same is_empty() idiom the function already uses for MMQ at 7739. That fallback needs ggml_vk_get_to_fp16, one of the seven gates here, so the registration is load-bearing for the fix rather than incidental.

Worth recording why my testing could not have caught this: the B390 is KHR_coopmat, which takes the other branch, so a clean 28/28 there says nothing about a device with VK_NV_cooperative_matrix2. Same trap shape as the one this PR fixes — a path advertised with no shader — one branch over.

@bri-prism bri-prism closed this Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant