Skip to content

Update SpeechRecognitionResult with audio timing attributes #192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alan33d wants to merge 1 commit into
base: main
Choose a base branch
from
+14 −0
Open

Update SpeechRecognitionResult with audio timing attributes #192

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,8 @@ This does not preclude adding support for this as a future API enhancement, and
<li>The user agent may also give the user a longer explanation the first time speech input is used, to let the user know what it is and how they can tune their privacy settings to disable speech recording if required.</li>

<li>To mitigate the risk of fingerprinting, user agents MUST NOT personalize speech recognition when performing speech recognition on a {{MediaStreamTrack}}.</li>

<li>To mitigate fingerprinting vectors associated with high-precision timing, user agents MUST apply timestamp fuzzing and precision reduction to {{SpeechRecognitionResult/audioStartTime}} and {{SpeechRecognitionResult/audioEndTime}} before exposing these attributes to scripts (e.g. by rounding to 2ms precision).</li>
</ol>

<h3 id="implementation-considerations">Implementation considerations</h3>
Expand Down Expand Up @@ -258,6 +260,8 @@ interface SpeechRecognitionResult {
readonly attribute unsigned long length;
getter SpeechRecognitionAlternative item(unsigned long index);
readonly attribute boolean isFinal;
readonly attribute DOMHighResTimeStamp? audioStartTime;
Comment thread
alan33d marked this conversation as resolved.
readonly attribute DOMHighResTimeStamp? audioEndTime;
};

// A collection of responses (used in continuous mode)
Expand Down Expand Up @@ -356,6 +360,16 @@ interface SpeechRecognitionPhrase {
</dd>
</dl>

<h4 id="speechrecoresult-attributes">SpeechRecognitionResult Attributes</h4>

<dl>
<dt><dfn attribute for=SpeechRecognitionResult>audioStartTime</dfn> attribute</dt>
<dd>A nullable {{DOMHighResTimeStamp}} representing the start of the audio segment corresponding to this recognition result, in milliseconds relative to the time origin. Returns null if the underlying recognition engine does not support audio segment start timestamps.</dd>

@hoch hoch Jul 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC SpeechSynthesisEvent uses elapsedTime, which is float.

Is there a specific design reason for using DOMHighResTimeStamp here?

@alan33d alan33d Jul 14, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No specific reason, I started with double? but it was suggested by @evanbliu to use DOMHighResTimeStamp? to mirror event.timeStamp property. I don't have a strong preference on this.

@evanbliu evanbliu Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, DOMHighResTimeStamp is just a typedef for a double but it makes it clear that the units are in milliseconds.

I assume SpeechSynthesisEvent uses float because DOMHighResTimeStamp didn't exist (or at least wasn't widely adopted) back in 2012.


<dt><dfn attribute for=SpeechRecognitionResult>audioEndTime</dfn> attribute</dt>
<dd>A nullable {{DOMHighResTimeStamp}} representing the end of the audio segment corresponding to this recognition result, in milliseconds relative to the time origin. Returns null if the underlying recognition engine does not support audio segment end timestamps.</dd>
</dl>

<p class=issue>The group has discussed whether WebRTC might be used to specify selection of audio sources and remote recognizers.
See <a href="https://lists.w3.org/Archives/Public/public-speech-api/2012Sep/0072.html">Interacting with WebRTC, the Web Audio API and other external sources</a> thread on public-speech-api@w3.org.</p>

Expand Down