SpeechToText Recognize from file - #3304
Open
VladislavAntonyuk wants to merge 2 commits into
Open
Conversation
VladislavAntonyuk
force-pushed
the
file-speech-to-text
branch
from
September 3, 2026 10:48
3a7cdc2 to
58060e9
Compare
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
It introduces a build-breaking global.json change (invalid JSON) and has correctness/API issues in the new recognition implementations (e.g., cancellation not wired; Tizen returns null for “unsupported”).
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a new RecognizeAsync(Stream, SpeechToTextOptions, CancellationToken) API to the SpeechToText essentials so callers can transcribe speech from an audio file/stream (in addition to the existing live “listen” flow), and wires it into the sample app UI.
Changes:
- Introduces
ISpeechToText.RecognizeAsync(...)plus corresponding wrappers inSpeechToTextandSpeechToTextImplementation. - Implements stream-based recognition for Windows (
System.Speech) and Android (pipe +RecognizerIntent.ExtraAudioSource), and adds a macOS/iOS stream-feed implementation usingSFSpeechAudioBufferRecognitionRequest. - Updates the sample page/view-model to pick a media file and display the recognized text.
File summaries
| File | Description |
|---|---|
| src/CommunityToolkit.Maui.Core/Essentials/SpeechToText/SpeechToTextImplementation.windows.cs | Adds stream-to-text recognition via SpeechRecognitionEngine. |
| src/CommunityToolkit.Maui.Core/Essentials/SpeechToText/SpeechToTextImplementation.tizen.cs | Adds a placeholder InternalRecognizeAsync stub for Tizen. |
| src/CommunityToolkit.Maui.Core/Essentials/SpeechToText/SpeechToTextImplementation.shared.cs | Exposes RecognizeAsync(...) on the main implementation type. |
| src/CommunityToolkit.Maui.Core/Essentials/SpeechToText/SpeechToTextImplementation.net.cs | Adds RecognizeAsync fallback throwing NotSupportedException. |
| src/CommunityToolkit.Maui.Core/Essentials/SpeechToText/SpeechToTextImplementation.android.cs | Adds Android stream transcription helper and InternalRecognizeAsync. |
| src/CommunityToolkit.Maui.Core/Essentials/SpeechToText/SpeechToText.shared.cs | Adds static SpeechToText.RecognizeAsync(...) convenience method. |
| src/CommunityToolkit.Maui.Core/Essentials/SpeechToText/SharedSpeechToTextImplementation.macios.cs | Implements stream recognition by feeding PCM buffers into Apple speech APIs. |
| src/CommunityToolkit.Maui.Core/Essentials/SpeechToText/OfflineSpeechToTextImplementation.shared.cs | Adds RecognizeAsync forwarding to the platform implementation. |
| src/CommunityToolkit.Maui.Core/Essentials/SpeechToText/ISpeechToText.shared.cs | Extends the interface with RecognizeAsync(...) and XML docs. |
| samples/CommunityToolkit.Maui.Sample/ViewModels/Essentials/SpeechToTextViewModel.cs | Adds a command to pick an audio file and call RecognizeAsync. |
| samples/CommunityToolkit.Maui.Sample/Pages/Essentials/SpeechToTextPage.xaml | Adds a button to trigger file recognition in the sample UI. |
| global.json | Comments out the SDK selection block. |
Review details
Suppressed comments (1)
src/CommunityToolkit.Maui.Core/Essentials/SpeechToText/SpeechToTextImplementation.android.cs:257
- Use
Trace.WriteLine()instead ofDebug.WriteLine()so the logging remains available in Release builds per toolkit guidelines.
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine($"Pipe streaming error: {ex}");
}
- Files reviewed: 11/11 changed files
- Comments generated: 12
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+75
to
+78
| var text = await speechToText.RecognizeAsync(audioStream, new SpeechToTextOptions(){Culture = CultureInfo.GetCultureInfo(CurrentLocale?.Language ?? defaultLanguage), | ||
| }, cancellationToken: token); | ||
| await Shell.Current.DisplayAlertAsync("Recognition Result", text, "OK"); | ||
| } |
Comment on lines
+62
to
+66
| /// <inheritdoc /> | ||
| public Task<string?> RecognizeAsync(Stream stream, SpeechToTextOptions options, CancellationToken cancellationToken = default) | ||
| { | ||
| return new SpeechToTextImplementation().RecognizeAsync(stream, options, cancellationToken); | ||
| } |
Comment on lines
+137
to
+139
| int bytesRead; | ||
| while ((bytesRead = await stream.ReadAsync(byteBuffer, 0, byteBuffer.Length).ConfigureAwait(false)) > 0) | ||
| { |
Comment on lines
+71
to
+75
| async Task<string?> InternalRecognizeAsync(System.IO.Stream stream, SpeechToTextOptions options, CancellationToken cancellationToken) | ||
| { | ||
| using var transcriber = new AudioStreamTranscriber(Application.Context); | ||
| return await transcriber.TranscribePcmStreamAsync(stream, language: Java.Util.Locale.ForLanguageTag(options.Culture.Name).ToLanguageTag()); | ||
| } |
Comment on lines
+221
to
+223
|
|
||
| tcs = new TaskCompletionSource<string>(); | ||
|
|
Comment on lines
+56
to
+58
| var sb = new StringBuilder(); | ||
| var tcs = new TaskCompletionSource<string>(); | ||
|
|
Comment on lines
+52
to
+54
| /// <remarks> | ||
| /// Speech recognition results will be surfaced via <see cref="RecognitionResultCompleted"/> | ||
| /// </remarks> |
Comment on lines
+23
to
+25
| /// <inheritdoc cref="ISpeechToText.StopListenAsync"/> | ||
| public static Task<string?> RecognizeAsync(Stream stream, SpeechToTextOptions options, CancellationToken cancellationToken = default) => | ||
| Default.RecognizeAsync(stream, options, cancellationToken); |
Comment on lines
1
to
+5
| using System.Diagnostics.CodeAnalysis; | ||
| using System.Net.Mime; | ||
| using Android.Content; | ||
| using Android.Media; | ||
| using Android.OS; |
Comment on lines
+199
to
+200
| public class AudioStreamTranscriber : Java.Lang.Object, IRecognitionListener | ||
| { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Change
Linked Issues
PR Checklist
approved(bug) orChampioned(feature/proposal)mainat time of PRAdditional information