feat(tts): add a Core ML Kokoro variant - #1384
Open
msluszniak wants to merge 2 commits into
Open
Conversation
Core ML cannot export Kokoro's token axis dynamically, so its models fix it and every chunk is padded up to that length. Add it as a second allowed spec next to the dynamic one, and wire the Core ML presets into the registry and the speech demo.
msluszniak
added
platform: ios
msluszniak
marked this pull request as ready for review
August 25, 2026 15:55
Same contract as the standard Core ML pair, so the entries just spread their XNNPACK twins and swap modelPaths, keeping the phonemizer and voices (pm_mateusz, df_anna) defined once. No app change: the Backend picker already gates on COREML_FP32 being present for the selected language, so both light up on their own.
msluszniak
requested review from
IgorSwat and
barhanc
and removed request for
IgorSwat
August 25, 2026 17:30
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a Core ML variant of Kokoro for iOS. Core ML cannot express the token axis dynamically, so its models fix it at 128 and every chunk is padded up to it; the frame axis stays dynamic (16..296), so durations and audio length are unchanged and no tempo scaling is involved.
The task accepts this as a second allowed spec (
padded) next to the existingdynamicone, and the padded branch collapsesminTokens/maxTokensonto the model's token count, so the existing worklet pads chunks with no other change. The synthesizer declares its frame range and the audio-length relation throughget_model_schema; the duration predictor is fully static.On an iPhone 16 the static-shape synthesizer measured 627 ms on Core ML against 4741 ms on XNNPACK (warm); on desktop the dynamic frame axis costs ~5% over static at full length and scales down with the frame count (486 ms at 296 frames, 181 ms at 120). Verified end to end on an iPhone 16 running this branch, with both models fetched from the published v0.10.0 tag. fp32 and CPU_ONLY: fp16 lands at 0.973 spectral correlation against eager versus 0.9993, and
ComputeUnit.ALLgave a 25 minute compile plus an execute failure.Introduces a breaking change?
Type of change
Tested on
Testing instructions
Run
apps/speech, open the Kokoro screen, pick Core ML in the Backend picker (iOS only, shown for the languages served by the standard weights) and synthesize. Compare against XNNPACK on the same text and voice.Screenshots
Related issues
Checklist