vulkan: register PTQ1_0 in the selection and supports_op enumerations - #149
vulkan: register PTQ1_0 in the selection and supports_op enumerations#149bri-prism wants to merge 1 commit into
Conversation
PTQ1_0's Vulkan pipelines are created but nothing can select them. Every PTQ1_0 reference in ggml-vulkan.cpp sits in pipeline creation (4637-5466) and none in the selection or support enumerations, so ggml_backend_vk_device_supports_op returns false, the scheduler never offers MUL_MAT to Vulkan, and the 54 generated SPIR-V shaders are unreachable. The build is green and the backend does nothing. Add `case GGML_TYPE_PTQ1_0:` alongside the existing GGML_TYPE_Q1_0 in the seven enumerations that have working PTQ1_0 pipelines behind them: ggml_vk_get_to_fp16 ggml_vk_get_mul_mat_mat_pipeline ggml_vk_get_dequantize_mul_mat_vec ggml_vk_get_mul_mat_mat_id_pipeline ggml_vk_get_dequantize_mul_mat_vec_id ggml_backend_vk_device_supports_op (MUL_MAT / MUL_MAT_ID gate) ggml_backend_vk_device_supports_op (GET_ROWS gate) There are twelve `case GGML_TYPE_Q1_0:` in this file; five are deliberately left alone. ggml_vk_get_cpy_pipeline (x2), GGML_OP_SET_ROWS and GGML_OP_DUP (x2) all depend on copy_to_quant.comp, which has no PTQ1_0 entry, so quantizing to PTQ1_0 on device is unimplemented. Advertising those paths would claim support for a shader that does not exist; a clean refusal is correct until copy_to_quant gains a PTQ1_0 entry. Measured on Intel Arc B390 (KHR_coopmat, proprietary Windows driver), test-backend-ops test -b Vulkan0 -o MUL_MAT, two runs, byte-identical: type OK not_supported FAIL q1_0 28 50 0 q2_0 28 50 0 ptq1_0 28 50 0 (was 0 OK / 78 not_supported) tq2_0 11 0 0 1037/1037 tests passed, 0 failures; total case count rises 1009 -> 1037. 28 is the full available set rather than a partial pass: the 50 declines are shape and permutation variants that q1_0 and q2_0 also decline on this device. The PTQ1_0 kernel itself needed no changes - it was correct, just unreachable.
|
Superseded by 705d9cc on My patch was incomplete and would have crashed on coopmat2 hardware. Registering PTQ1_0 in The fix returns Worth recording why my testing could not have caught this: the B390 is KHR_coopmat, which takes the other branch, so a clean 28/28 there says nothing about a device with |
Stacked on #148. Without this, PTQ1_0's Vulkan backend builds green and executes nothing.
The problem
Every PTQ1_0 reference in
ggml-vulkan.cppsits in pipeline creation (lines 4637–5466). None are in the selection or support enumerations. Soggml_backend_vk_device_supports_opreturnsfalse, the scheduler never offersMUL_MATto Vulkan, and the 54 generated SPIR-V shaders are unreachable code.This is the "registers but is never selected" failure mode, and it is worse than a visible failure because every signal reads healthy — clean build, shaders generated,
test-backend-opsreporting1009/1009 ... Backend Vulkan0: OK. On inspection all 78 ptq1_0 lines saidnot supported [Vulkan0].The fix
case GGML_TYPE_PTQ1_0:alongside the existingGGML_TYPE_Q1_0in the seven enumerations that have working PTQ1_0 pipelines behind them:ggml_vk_get_to_fp16ggml_vk_get_mul_mat_mat_pipelineggml_vk_get_dequantize_mul_mat_vecggml_vk_get_mul_mat_mat_id_pipelineggml_vk_get_dequantize_mul_mat_vec_idggml_backend_vk_device_supports_opggml_backend_vk_device_supports_opFive sites deliberately left alone
There are twelve
case GGML_TYPE_Q1_0:in this file, not seven. Do not blanket-add:ggml_vk_get_cpy_pipeline(×2)supports_op→GGML_OP_SET_ROWSsupports_op→GGML_OP_DUP(×2)All five depend on
copy_to_quant.comp, which has no PTQ1_0 entry — quantizing to PTQ1_0 on device is unimplemented. Adding them would advertise a path whose shader does not exist, which is the inverse of the coopmat2 break fixed in #148 (there, a shader generated with no decoder; here, a path advertised with no shader). A clean refusal is correct untilcopy_to_quantgains a PTQ1_0 entry.Measured
Intel Arc B390, KHR_coopmat, proprietary Windows driver.
test-backend-ops test -b Vulkan0 -o MUL_MAT, two runs, ptq1_0 per-case output byte-identical:1037/1037 tests passed, 0 failures. Before this patch: ptq1_0 was 0 OK / 78 not supported.Two checks worth stating explicitly, because a green run proved nothing here twice already:
OKcounts, not line counts.grep -c type_a=ptq1_0counts the skip lines too; that is how the pre-patch state read as a pass.28 is the full available set, not a partial pass — the 50 declines are shape and permutation variants that q1_0 and q2_0 also decline on this device.
The PTQ1_0 kernel itself needed no changes. It was correct; it was unreachable. Mirroring Q1_0 rather than Q2_0 was the right call.