metal: fix the PTQ1_0 CPY crash and add the missing PTQ1_0 dequant-copy kernels - #162
Merged
Conversation
…py kernels
supports_op accepted CPY in both directions for PTQ1_0 while no cpy kernel for
the type was instantiated, so the first such op asked for a pipeline that does
not exist ("kernel not found in any metal library") and the process died with
SIGSEGV. test-backend-ops -o CPY on Metal therefore exited 139 partway through
the type list while every line it did print was OK.
- PTQ1_0 -> F32/F16: instantiate kernel_cpy_ptq1_0_f32/f16 on the existing
cpy_q_f32 template with dequantize_ptq1_0 (nl = 8, same as PQ2_0 and the
PTQ1_0 get_rows/mul_mm instantiations).
- F32 -> PTQ1_0: decline in supports_op so it falls back to the CPU; there is
no Metal quantize_ptq1_0.
test-backend-ops test -b MTL0 -o CPY: exit 0, 203/203 passed (was exit 139).
Both PTQ1_0 -> F32 cases OK, the 15 F32 -> PTQ1_0 cases report not supported.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Metal accepted
GGML_OP_CPYin both directions forPTQ1_0while no copy kernel for the type existed. The first such op asked the library for a pipeline that is not there (kernel not found in any metal library: kernel_cpy_f32_ptq1_0) and the process died with SIGSEGV.PTQ1_0 -> F32/F16: instantiatekernel_cpy_ptq1_0_f32andkernel_cpy_ptq1_0_f16on the existingcpy_q_f32template withdequantize_ptq1_0(nl = 8, matching thePQ2_0copy and thePTQ1_0get_rows and mul_mm instantiations).F32 -> PTQ1_0: decline insupports_opso the op falls back to the CPU. There is no Metalquantize_ptq1_0.Why
Same permissive-gate pattern as the per-expert mat-vec crash fixed in #157:
supports_opclaimed a type the shader library never instantiated. It also made the CPY sweep intest-backend-opslook green: the run aborted at the firstPTQ1_0case, after thePQ2_0cases, and everything it had printed was OK.Verification
test-backend-ops test -b MTL0 -o CPYon an M5 Pro:PTQ1_0 -> F32casesF32 -> PTQ1_0casesNo kernel used by the model graph changes; this only adds instantiations and removes a false capability claim.