Stream and Channel Mapping Support by reinecke · Pull Request #2015 · AcademySoftwareFoundation/OpenTimelineIO

reinecke · 2026-04-04T00:09:26Z

Link the Issue(s) this Pull Request is related to.

If there is an associated issue, link it in the form:

Fixes #145

Summarize your change.

To for an explanation of the implementation, check out the docs.

This PR implements the proposal that was created to address #145 and add the ability to do stream/channel addressing and mapping from sources into the composition.

The C++ implementation of the mappings was built following the pattern of Clip.media_references.

Reference associated tests.

Unittests were added for the new features.

reinecke · 2026-04-04T00:16:37Z

@mrlimbic here is my first pass at stream/channel mapping. Please feel free to provide feedback here.

codecov-commenter · 2026-04-04T00:22:06Z

Codecov Report

❌ Patch coverage is 91.91050% with 47 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.49%. Comparing base (6e13de4) to head (b3bae93).

Files with missing lines	Patch %	Lines
...entimelineio-bindings/otio_serializableObjects.cpp	80.00%	23 Missing ⚠️
src/opentimelineio/streamInfo.h	60.00%	2 Missing ⚠️
...timelineio/opentimelineio/algorithms/track_algo.py	0.00%	2 Missing ⚠️
...melineio/opentimelineio/schema/audio_mix_matrix.py	75.00%	2 Missing ⚠️
...neio/opentimelineio/schema/index_stream_address.py	75.00%	2 Missing ⚠️
...timelineio/opentimelineio/schema/stream_address.py	75.00%	2 Missing ⚠️
...neio/schema/stream_channel_index_stream_address.py	75.00%	2 Missing ⚠️
...pentimelineio/opentimelineio/schema/stream_info.py	75.00%	2 Missing ⚠️
...ntimelineio/opentimelineio/schema/stream_mapper.py	75.00%	2 Missing ⚠️
...imelineio/opentimelineio/schema/stream_selector.py	75.00%	2 Missing ⚠️
... and 4 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2015      +/-   ##
==========================================
+ Coverage   85.14%   85.49%   +0.34%     
==========================================
  Files         181      205      +24     
  Lines       12780    13354     +574     
  Branches     1206     1217      +11     
==========================================
+ Hits        10882    11417     +535     
- Misses       1715     1753      +38     
- Partials      183      184       +1

Flag	Coverage Δ
py-unittests	`85.49% <91.91%> (+0.34%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/opentimelineio/audioMixMatrix.cpp	`100.00% <100.00%> (ø)`
src/opentimelineio/indexStreamAddress.cpp	`100.00% <100.00%> (ø)`
src/opentimelineio/indexStreamAddress.h	`100.00% <100.00%> (ø)`
src/opentimelineio/mediaReference.cpp	`100.00% <100.00%> (ø)`
src/opentimelineio/mediaReference.h	`100.00% <ø> (ø)`
src/opentimelineio/streamAddress.cpp	`100.00% <100.00%> (ø)`
...opentimelineio/streamChannelIndexStreamAddress.cpp	`100.00% <100.00%> (ø)`
...c/opentimelineio/streamChannelIndexStreamAddress.h	`100.00% <100.00%> (ø)`
src/opentimelineio/streamInfo.cpp	`100.00% <100.00%> (ø)`
src/opentimelineio/streamMapper.cpp	`100.00% <100.00%> (ø)`
... and 22 more

... and 3 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6e13de4...b3bae93. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Eric Reinecke <ereinecke@netflix.com>

darbyjohnston

Overall the C++ code looks fine, it seems to follow all of our conventions, I just left a few minor comments. Thanks for including the Doxygen markup and export macros.

I only skimmed the Python code, it would be good to get someone with more Python experience to take a look.

At some point it might be nice to add C++ tests, but for now the Python tests exercise the functionality.

darbyjohnston · 2026-04-22T15:57:24Z

+
+## Overview
+
+OTIO's stream and channel mapping support allows media references to describe the individual streams they contain (video eyes, audio channels, etc.) and for timeline items to select or remix those streams via effects. The system has three layers: **addressing** a stream within a container, **describing** a stream's semantic role, and **selecting or mixing** streams on a clip.


I haven't seen "video eyes" before, is that a convention?

darbyjohnston · 2026-04-22T15:58:40Z

+2. **Additional streams** SHOULD prefix an `Identifier` value to form a unique key within the media reference.
+
+   ```python
+   # A music stem alongside the audio mix


I thought this was a typo ('stem' -> 'stream'), but I guess stem is an audio term?

darbyjohnston · 2026-04-22T16:11:20Z

+| Class | Use when |
+|---|---|
+| `IndexStreamAddress` | The container uses a single integer index per stream (e.g. WAV channel index, ffmpeg stream index) |
+| `StreamChannelIndexStreamAddress` | The container organises media into tracks, each with one or more channels (e.g. MP4/MOV, MXF) |


Could we simplify this class name a bit, maybe ChannelStreamAddress?

darbyjohnston · 2026-04-22T16:18:51Z

+    /// @brief This struct provides the AudioMixMatrix schema.
+    struct Schema
+    {
+        static char constexpr name[]  = "AudioMixMatrix";


Thanks, I like the explicit type here vs. the static auto constexpr name we have in the rest of the headers. :)

darbyjohnston · 2026-04-22T16:27:28Z

+    OTIO_API explicit IndexStreamAddress(int64_t index = 0);
+
+    /// @brief Return the stream index.
+    OTIO_API int64_t index() const noexcept { return _index; }


I'm not sure if the export macros are needed on inline functions, we should double check that.

darbyjohnston · 2026-04-22T16:46:33Z

+
+    using Parent = SerializableObject;
+
+    OTIO_API StreamAddress();


We could use = default here to save a bit of code since the constructor is empty. Just a suggestion though, I'm not sure we use default anywhere else yet. At some point we could also do a separate pass over the code replacing empty ctors/dtors with default.

darbyjohnston · 2026-04-22T17:00:21Z

+/// @brief An effect that selects specific named output streams from an item.
+///
+/// Use this to select a stereo view, specific audio channels, etc.
+/// The clip will expose these streams out with the same naming.


Minor, but I thought the wording on this was a little confusing:

The clip will expose these streams out with the same naming.

Maybe:

The stream names match the names within the clip.

darbyjohnston · 2026-04-22T17:02:33Z

+
+    using Parent = StreamAddress;
+
+    OTIO_API explicit StringStreamAddress(std::string const& address = std::string());


Minor, but we could add a Doxygen comment here for completeness:

/// @brief Create a new StringStreamAddress.

darbyjohnston · 2026-04-22T18:03:27Z

-from . import (
-    track_algo
-)
+from . import track_algo


It looks like the changes in timeline_algo.py and track_algo.py are just formatting?

darbyjohnston · 2026-04-22T18:10:51Z

+
+Stream effects are `Effect` subclasses that attach to a clip's `effects` list and transform which streams are visible downstream and under what names. They operate on the stream identifiers declared in `available_streams`.
+
+### StreamSelector


Just curious if we need the StreamSelector class, it seems like you could do the same thing with StreamMapper?

darbyjohnston · 2026-04-23T20:44:53Z

Just to continue the discussion from today's OTIO meeting:

If multiple media references expose different stream information, how is that handled by the stream effects?

For example, say you have a clip with two media references, one with 5.1 audio and the other with mono:

Clip
- Media Reference: "5.1"
  - Stream Info: "left"
  - Stream Info: "surround_center_front"
  - ...
- Media Reference: "mono"
  - Stream Info: "monaural"

How would you use an AudioMixMatrix effect to make the clip stereo? Either use one effect with all of the streams:

AudioMixMatrix
- "left"
  - "left", 1.0
  - "surround_center_front", 0.707
  - ...
  - "monaural", 1.0
- "right"
  - "right", 1.0
  - "surround_center_front", 0.707
  - ...
  - "monaural", 1.0

Or use multiple AudioMixMatrix effects?

AudioMixMatrix
- "left"
  - "left", 1.0
  - "surround_center_front", 0.707
  - ...
- "right"
  - "right", 1.0
  - "surround_center_front", 0.707
  - ...
AudioMixMatrix
- "left"
  - "monaural", 1.0
- "right"
  - "monaural", 1.0

reinecke added 8 commits April 7, 2026 17:49

Initial pass implementing stream/channel mapping

ba34acf

Signed-off-by: Eric Reinecke <ereinecke@netflix.com>

Added StreamChannelIndexStreamAddress

16d2b78

Signed-off-by: Eric Reinecke <ereinecke@netflix.com>

Added StreamMapper effect

9c866cc

Signed-off-by: Eric Reinecke <ereinecke@netflix.com>

Updated constructors and reprs to better match expectations

3cfc7f3

Signed-off-by: Eric Reinecke <ereinecke@netflix.com>

Added/updated docs and version map

e08d0cd

Signed-off-by: Eric Reinecke <ereinecke@netflix.com>

Fixed linting

c6d9646

Signed-off-by: Eric Reinecke <ereinecke@netflix.com>

Fixed sphinx documentation warnings

d531ed0

Signed-off-by: Eric Reinecke <ereinecke@netflix.com>

Fixed linting issues

b3bae93

Signed-off-by: Eric Reinecke <ereinecke@netflix.com>

reinecke force-pushed the feature/stream_channel_mapping_support branch from 0d6c0cf to b3bae93 Compare April 7, 2026 22:50

darbyjohnston requested changes Apr 22, 2026

View reviewed changes


		## Overview

		OTIO's stream and channel mapping support allows media references to describe the individual streams they contain (video eyes, audio channels, etc.) and for timeline items to select or remix those streams via effects. The system has three layers: addressing a stream within a container, describing a stream's semantic role, and selecting or mixing streams on a clip.


		using Parent = StreamAddress;

		OTIO_API explicit StringStreamAddress(std::string const& address = std::string());


		Stream effects are `Effect` subclasses that attach to a clip's `effects` list and transform which streams are visible downstream and under what names. They operate on the stream identifiers declared in `available_streams`.

		### StreamSelector

Conversation

reinecke commented Apr 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

reinecke commented Apr 4, 2026

Uh oh!

codecov-commenter commented Apr 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

darbyjohnston left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

darbyjohnston commented Apr 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

reinecke commented Apr 4, 2026 •

edited

Loading

codecov-commenter commented Apr 4, 2026 •

edited

Loading