MilkdropPreset: add opt-in preprocessed-HLSL cache hooks (targeting 4.3)#1011
Draft
keeper-of-memes wants to merge 1 commit into
Draft
Conversation
The Milkdrop shader transpiler runs the hlslparser text preprocessor (macro expansion) over each preset shader before parsing it. That step is pure text and deterministic for a given input, and it dominates the CPU cost of a preset load on slower devices (~100 ms of a ~177 ms transpile measured on an Apple TV 4K / A15 in a release build). This adds an opt-in application-provided cache for that step: - projectm_preprocess_cache_hooks (get/put/user) and projectm_set_preprocess_cache() in the public C API. The hooks are copied by value; passing NULL clears them. With no hooks registered, behaviour is byte-identical to before. - The pure-text assembly of the preprocessor input is extracted verbatim from MilkdropShader::PreprocessPresetShader into the GL-free free function AssembleApplyPreprocessorInput(), so external tooling (e.g. a build-time precompute step for a bundled preset pack) can produce byte-identical preprocessor input without a GL context, textures, or PresetState. - ComputePreprocessCacheKey() derives a version-salted, collision-safe key (two independent 64-bit FNV-1a hashes plus exact length) from the exact bytes handed to ApplyPreprocessor. The salt is bumped whenever the assembly, static shader header, or hlslparser changes, so stale entries simply miss and fall back to the live preprocessor, never producing wrong output. - TranspileHLSLShader() consults the hooks before ApplyPreprocessor (a hit skips it entirely) and stores the result after a miss. All hook invocations happen on the thread that calls the projectM rendering functions, so no additional synchronisation is required. Measured on tvOS/ANGLE with a prepopulated cache: preset-load transpile cost drops from ~177 ms to ~77 ms (the residual being parse/generate and the GL compile).
keeper-of-memes
marked this pull request as ready for review
July 6, 2026 14:54
keeper-of-memes
marked this pull request as draft
July 6, 2026 14:54
Author
|
Update from production use: the cache is now deployed on Apple TV HD (A8) hardware as well, where it matters considerably more than on the A15 it was measured on — cold-cache preset loads on the A8 run into whole seconds, and the prepopulated cache absorbs the preprocessor share of that on first load. Also note this branch already includes the fix for computing the cache key when only the put hook is registered (previously a put-only registration would store under an empty key). Still targeting 4.3; will rebase and move the API into a dedicated header once #970 lands. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Milkdrop shader transpiler runs the hlslparser text preprocessor (macro expansion) over each preset shader before parsing it. On slower devices this step dominates the CPU cost of a preset load: ~100 ms of a ~177 ms transpile measured on an Apple TV 4K / A15 (release build), which is a guaranteed dropped-frames hitch wherever the load runs on the GL thread.
The preprocessor's output is a pure, deterministic function of its input text (verified with a golden-output harness: byte-identical output and identical downstream GLSL), so it can be cached across runs, and even precomputed at build time for a bundled preset pack.
Change
projectm_preprocess_cache_hooks(get/put/user) plusprojectm_set_preprocess_cache(). The struct is copied by value; NULL clears. projectM stores no cache itself, so persistence and policy are entirely the application's. With no hooks registered, behaviour is byte-identical to before.MilkdropShader::PreprocessPresetShaderinto the GL-free free functionAssembleApplyPreprocessorInput()(newMilkdropShaderPreprocess.{cpp,hpp}), so external tooling (e.g. a build-time cache generator) can produce byte-identical preprocessor input without a MilkdropShader instance, GL context, textures, or PresetState.ComputePreprocessCacheKey()hashes the exact bytes handed toApplyPreprocessor: two independent 64-bit FNV-1a passes plus the exact length (~128 bits effective), prefixed with a compile-time version salt. The salt is bumped whenever the assembly transform, static shader header, or hlslparser changes, so stale entries simply miss and fall back to the live preprocessor. Wrong output is impossible; the worst case is a cache miss.TranspileHLSLShader()consults the hooks beforeApplyPreprocessor(a hit skips it entirely) and stores after a miss. Hook invocations happen on the thread that calls the projectM rendering functions, so no extra synchronisation is needed.Measurements
With a build-time prepopulated cache for a bundled preset pack (tvOS/ANGLE, A15): preset-load transpile 177 → ~77 ms. The residual is parse/generate and the GL compile, not the preprocessor.
Notes for reviewers
RenderContextgains one pointer field to propagate the hooks from the instance to the shader code.MilkdropShader.cppdiff is dominated by the verbatim code move intoMilkdropShaderPreprocess.cpp.