Inline FP64 emulation when a kernel requires SIMD32 on fused-EU platforms - #428
Open
pvelesko wants to merge 2 commits into
Open
Inline FP64 emulation when a kernel requires SIMD32 on fused-EU platforms#428pvelesko wants to merge 2 commits into
pvelesko wants to merge 2 commits into
Conversation
…es SIMD32 On platforms without native FP64, PreCompiledFuncImport rewrites double arithmetic into __igcbuiltin_dp_* calls and keeps the slow DP emulation builtins as subroutines/stack calls rather than inlining them. IGC otherwise keeps calls out of SIMD32 on platforms that need the fused-EU call workaround, through ForceLowestSIMDForStackCalls and the requireCallWA() SIMD32 bail-out in COpenCLKernel::checkSIMDCompileConds(). Both guards sit inside "if (requiredSimdSize == 0)", so a kernel that requires SIMD32 through intel_reqd_sub_group_size(32) or SPIR-V OpExecutionMode SubgroupSize 32 bypasses them and reaches SIMD32 codegen with the emulation still in call form. The kernel added here does exactly that: intel_reqd_sub_group_size(32) with rsqrt() on doubles under -cl-fp64-gen-emu on dg2. The test dumps the generated assembly and requires that the kernel really is generated and that neither a stack call nor a surviving __igcbuiltin_dp_ call appears anywhere in the dump. Those two absence patterns are given to FileCheck as --implicit-check-not rather than written as leading CHECK-NOT directives: a CHECK-NOT placed before the first positive CHECK only guards the input ahead of that match, which is not where the emulation calls are printed. Without the following commit the emulation stays in call form and the generated code is silently wrong: the double returned by an emulated call is replaced by an unrelated live value of the caller. Signed-off-by: Paulius Velesko <pvelesko@pglc.io>
…orms On platforms without native FP64, PreCompiledFuncImport rewrites double arithmetic into __igcbuiltin_dp_* calls and keeps the slow DP emulation builtins as subroutines/stack calls rather than inlining them. IGC otherwise keeps calls out of SIMD32 on platforms that need the fused-EU call workaround, through ForceLowestSIMDForStackCalls and the requireCallWA() SIMD32 bail-out in COpenCLKernel::checkSIMDCompileConds(). Both guards sit inside "if (requiredSimdSize == 0)", so a kernel that requires SIMD32 through intel_reqd_sub_group_size(32) or SPIR-V OpExecutionMode SubgroupSize 32 bypasses them and reaches SIMD32 codegen with the emulation still in call form. The generated code is then silently wrong: the double returned by an emulated call is replaced by an unrelated live value of the caller, so for instance rsqrt() yields the value of another local. Measured on Gen12LP, every lane of the result is wrong, and forcing the emulation inline via EmulationFunctionControl makes the same kernel bit-exact. Keep the emulation inlined for such kernels instead. SIMD8 and SIMD16, and any explicit EmulationFunctionControl request, are unaffected. Fixes intel#397 Signed-off-by: Paulius Velesko <pvelesko@pglc.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On platforms without native FP64,
PreCompiledFuncImportrewrites double arithmetic into__igcbuiltin_dp_*calls and keeps the slow DP emulation builtins in call form. On platforms needing the fused-EU call workaround that is unsafe at SIMD32, and the guards that would otherwise prevent it do not apply to a kernel that requires SIMD32:OpenCLKernelCodeGen.cpp:2669(hasSubroutine) and:2849(forceLowestSIMDForStackCalls) both sit insideif (requiredSimdSize == 0).:2687, which forces SIMD16 when optimizations are disabled, carries the same condition, so-cl-opt-disabledoes not avoid it either.:2829does handle a required SIMD32 by disabling EU fusion and warning, but only whenhasNestedCall || hasIndirectCall || isIndirectGroup. Plain emulation subroutines match none of those.So a kernel with
intel_reqd_sub_group_size(32)(or SPIR-VOpExecutionMode SubgroupSize 32) reaches SIMD32 codegen with the emulation still in call form, and the generated code is wrong with no diagnostic: the double returned by an emulated call is replaced by an unrelated live value of the caller, sorsqrt()yields another local. Substituting that value into a host model reproduces all 32 GPU lanes bit exactly.Keep the emulation inlined for those kernels instead, unless call form was explicitly requested through
EmulationFunctionControl.Measured on the test kernel, same source and same ocloc, only libigc differing:
__igcbuiltin_dp_calls in the asm dumpParts with native FP64 (Battlemage, PVC) generate no emulation calls at all and are unaffected, which is why the same reproducer passes there.
An alternative would be to widen the
:2829case to cover plain stack calls, disabling EU fusion for the whole kernel and warning. I chose inlining because it is local to the pass that creates the calls and does not change a global hardware mode as a side effect, but I am happy to take the other approach if preferred.Note that
IGC/ocloc_testsonly builds with-DIGC_OPTION__ENABLE_OCLOC_LIT_TESTS=ON.Fixes #397