From c1284a263ef9d9a401d72f57f3504e94c4d82df6 Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Mon, 13 Jul 2026 11:04:00 +0200 Subject: [PATCH 01/19] Removing developer preview warning --- README.md | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/README.md b/README.md index 585848a7..f5635885 100644 --- a/README.md +++ b/README.md @@ -21,19 +21,7 @@ # LiveKit Unity SDK - - -⚠️ Warning - - -> This SDK is currently in Developer Preview mode and not ready for production use. There will be bugs and APIs may change during this period. - -> - -> We welcome and appreciate any feedback or contributions. You can create issues here or chat live with us and other users in our community at https://community.livekit.io/. - - From a9440204826e62953c232447a740333739f8e2a5 Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Mon, 13 Jul 2026 11:19:25 +0200 Subject: [PATCH 02/19] Adding a screenshot of meet --- .github/Meet_Screenshot.png | 3 +++ Samples~/Meet/README.md | 5 +++++ 2 files changed, 8 insertions(+) create mode 100644 .github/Meet_Screenshot.png create mode 100644 Samples~/Meet/README.md diff --git a/.github/Meet_Screenshot.png b/.github/Meet_Screenshot.png new file mode 100644 index 00000000..3766cccd --- /dev/null +++ b/.github/Meet_Screenshot.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f46e66cfc7eb9d139e45a329551fcdc8bdfdb830f6dfccae5e2f93f389add0a6 +size 312525 diff --git a/Samples~/Meet/README.md b/Samples~/Meet/README.md new file mode 100644 index 00000000..959e0ca3 --- /dev/null +++ b/Samples~/Meet/README.md @@ -0,0 +1,5 @@ +## Meet sample + + + +Screenshot of the Meet sample \ No newline at end of file From 8064639fbe03d3eed2cce5d8a78fa96c824d87c4 Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Mon, 13 Jul 2026 11:20:49 +0200 Subject: [PATCH 03/19] other image path --- Samples~/Meet/README.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/Samples~/Meet/README.md b/Samples~/Meet/README.md index 959e0ca3..c926c79c 100644 --- a/Samples~/Meet/README.md +++ b/Samples~/Meet/README.md @@ -1,5 +1,3 @@ ## Meet sample - - -Screenshot of the Meet sample \ No newline at end of file +Screenshot of the Meet sample \ No newline at end of file From e3042024a5e8f06f099acc978d95c8e336a74706 Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Mon, 13 Jul 2026 11:50:32 +0200 Subject: [PATCH 04/19] Adding youtube tutorial screenshot, adding samples links --- ...reenshot.png => meet_sample_screenshot.png} | 0 .github/youtube_tutorial_screenshot.png | 3 +++ README.md | 18 ++++++++++++------ Samples~/Meet/README.md | 2 +- 4 files changed, 16 insertions(+), 7 deletions(-) rename .github/{Meet_Screenshot.png => meet_sample_screenshot.png} (100%) create mode 100644 .github/youtube_tutorial_screenshot.png diff --git a/.github/Meet_Screenshot.png b/.github/meet_sample_screenshot.png similarity index 100% rename from .github/Meet_Screenshot.png rename to .github/meet_sample_screenshot.png diff --git a/.github/youtube_tutorial_screenshot.png b/.github/youtube_tutorial_screenshot.png new file mode 100644 index 00000000..a5fe808a --- /dev/null +++ b/.github/youtube_tutorial_screenshot.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0b6dffa6b3e952327e386cc1213bc7a1ca517b502d7c2c128da5a40fb8943b01 +size 3138615 diff --git a/README.md b/README.md index f5635885..4fabe481 100644 --- a/README.md +++ b/README.md @@ -25,18 +25,19 @@ -Use this SDK to add realtime video, audio and data features to your Unity app. By connecting to LiveKit Cloud or a self-hosted server, you can quickly build applications such as multi-modal AI, live streaming, or video calls with just a few lines of code. +Use this SDK to add realtime video, audio and data features to your Unity app. By connecting to LiveKit Cloud or a self-hosted server, you can quickly build applications such as multi-modal AI like voice AI agents for NPCs, live streaming, or video calls with just a few lines of code. - - [SDK Reference »](https://livekit.github.io/client-sdk-unity) - +Screenshot of the LiveKit Unity SDK Youtube tutorial + +[Youtube Agents NPC Tutorial](https://www.youtube.com/@livekit_io) ## Platform Support +We officially support Unity 2022 onwards, we test on Unity 2022.3.62 and Unity 6000.3.10. - [x] Windows @@ -51,11 +52,9 @@ Use this SDK to add realtime video, audio and data features to your Unity app. B - [ ] WebGL - We plan to support all Unity platforms with this SDK. WebGL is currently supported with [client-sdk-unity-web](https://github.com/livekit/client-sdk-unity-web). - ## Installation @@ -84,6 +83,13 @@ If you want to use tagged release versions, use `https://github.com/livekit/clie The package is also hosted in the OpenUPM package registry. Here is the guide on how to use the OpenUPM registry to import the package: https://openupm.com/packages/io.livekit.livekit-sdk/#modal-manualinstallation +## Samples + +The repo contains these sample projects: +- [Meet](https://github.com/livekit/client-sdk-unity/tree/main/Samples~/Meet) +- [Agents](https://github.com/livekit/client-sdk-unity/tree/main/Samples~/Agents) + + ## Local Development diff --git a/Samples~/Meet/README.md b/Samples~/Meet/README.md index c926c79c..fcecb9f9 100644 --- a/Samples~/Meet/README.md +++ b/Samples~/Meet/README.md @@ -1,3 +1,3 @@ ## Meet sample -Screenshot of the Meet sample \ No newline at end of file +Screenshot of the Meet sample \ No newline at end of file From 55aaf11b799f1e91120b2e6ac12eb8acca3c7ef0 Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Mon, 13 Jul 2026 11:51:39 +0200 Subject: [PATCH 05/19] Adding tutorial section --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 4fabe481..7222f801 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,8 @@ Use this SDK to add realtime video, audio and data features to your Unity app. B [SDK Reference »](https://livekit.github.io/client-sdk-unity) +## Tutorial + Screenshot of the LiveKit Unity SDK Youtube tutorial [Youtube Agents NPC Tutorial](https://www.youtube.com/@livekit_io) From 0cccc1343a5017dd0a2fcde046f844c60d14f9e2 Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Mon, 13 Jul 2026 12:44:12 +0200 Subject: [PATCH 06/19] More meet readme --- .github/meet_sample_screenshot.png | 4 ++-- Samples~/Meet/README.md | 20 +++++++++++++++++++- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/meet_sample_screenshot.png b/.github/meet_sample_screenshot.png index 3766cccd..b5c32b1f 100644 --- a/.github/meet_sample_screenshot.png +++ b/.github/meet_sample_screenshot.png @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:f46e66cfc7eb9d139e45a329551fcdc8bdfdb830f6dfccae5e2f93f389add0a6 -size 312525 +oid sha256:816d49d248767098776a2f2834c4ddeb110247fec7c0b23878f0782c908b6379 +size 826744 diff --git a/Samples~/Meet/README.md b/Samples~/Meet/README.md index fcecb9f9..e9ce70e0 100644 --- a/Samples~/Meet/README.md +++ b/Samples~/Meet/README.md @@ -1,3 +1,21 @@ ## Meet sample -Screenshot of the Meet sample \ No newline at end of file +Screenshot of the Meet sample + +## Getting started + +### Project setup + +The sample can either be imported via the package manager to get access to the assets or opened as a full Unity project. The project is set up to build and run on all supported platforms. + +### Token source + +In order to connect to your LiveKit server, configure the token source component config used in the scene. To learn more about tokens, see https://docs.livekit.io/frontends/build/authentication. + +### Audio system + +The LiveKit Unity SDK offers two audio systems. The Unity audio path uses the Unity APIs for audio input and output. Platform Audio is the alternative, where the native LiveKit plugin manages audio input and output. You can select which path to use on the MeetManager component. + +### Common sample package + +In order to get access to common sample functions like the on device scrolling log, make sure to import the [Common](https://github.com/livekit/client-sdk-unity/tree/main/Samples~/Common) sample from the LiveKit Unity Package in the package manager. \ No newline at end of file From 7146ee2ff57a437e69c28a6c62ebee2217ada152 Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Mon, 13 Jul 2026 12:53:22 +0200 Subject: [PATCH 07/19] Adding agents sample readme --- Samples~/Agents/README.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 Samples~/Agents/README.md diff --git a/Samples~/Agents/README.md b/Samples~/Agents/README.md new file mode 100644 index 00000000..219e3970 --- /dev/null +++ b/Samples~/Agents/README.md @@ -0,0 +1,23 @@ +## Agents sample + +## Tutorial + +Screenshot of the LiveKit Unity SDK Youtube tutorial + +[Youtube Agents NPC Tutorial](https://www.youtube.com/@livekit_io) + +## Getting started + +### Project setup + +The sample can either be imported via the package manager to get access to the assets or opened as a full Unity project. The project is set up to build and run on all supported platforms. + +### Agent dispatch + +To talk to an agent, the app needs a token to connect to a LiveKit room and dispatch an agent. The project is already configured to automatically connect to the LiveKit homepage agent you can try at www.livekit.com. + +To connect to your own LiveKit server, edit the token source component config with your projects token source. + +### Common sample package + +In order to get access to common sample functions like the on device scrolling log, make sure to import the [Common](https://github.com/livekit/client-sdk-unity/tree/main/Samples~/Common) sample from the LiveKit Unity Package in the package manager. \ No newline at end of file From a19fa127bb630ae229393eed10e7edb323a076c2 Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Mon, 13 Jul 2026 12:55:01 +0200 Subject: [PATCH 08/19] Add description text --- Samples~/Agents/README.md | 2 ++ Samples~/Meet/README.md | 2 ++ 2 files changed, 4 insertions(+) diff --git a/Samples~/Agents/README.md b/Samples~/Agents/README.md index 219e3970..c63ac74a 100644 --- a/Samples~/Agents/README.md +++ b/Samples~/Agents/README.md @@ -1,5 +1,7 @@ ## Agents sample +Connect to a voice AI agent. + ## Tutorial Screenshot of the LiveKit Unity SDK Youtube tutorial diff --git a/Samples~/Meet/README.md b/Samples~/Meet/README.md index e9ce70e0..4ffae2cc 100644 --- a/Samples~/Meet/README.md +++ b/Samples~/Meet/README.md @@ -1,5 +1,7 @@ ## Meet sample +Connect to a video call with other remote participants. + Screenshot of the Meet sample ## Getting started From beb7d34f35d780eb0c0e1e434a2b1be13ccdfd8f Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Tue, 14 Jul 2026 14:14:21 +0200 Subject: [PATCH 09/19] Removing duplicate samples mention --- README.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/README.md b/README.md index 7222f801..e53e13b9 100644 --- a/README.md +++ b/README.md @@ -207,10 +207,6 @@ If your project disables package editor scripts or uses a custom Xcode export pi ## Examples - - -Use the samples of the package to see how to use the SDK. - ### Tokens You need a token to join a LiveKit room as a participant. Read more about tokens here: https://docs.livekit.io/frontends/reference/tokens-grants/ From 902ac8ef6a577381d41bb7fd7a5288780dc9df9f Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Wed, 15 Jul 2026 08:39:05 +0200 Subject: [PATCH 10/19] Less white space, put async to bottom, new snippet for connect --- README.md | 246 ++++++++++++++++-------------------------------------- 1 file changed, 71 insertions(+), 175 deletions(-) diff --git a/README.md b/README.md index e53e13b9..e08cd2d4 100644 --- a/README.md +++ b/README.md @@ -1,28 +1,16 @@ - - - - - The LiveKit icon, the name of the repository and some sample code in the background. - - - - - # LiveKit Unity SDK - - Use this SDK to add realtime video, audio and data features to your Unity app. By connecting to LiveKit Cloud or a self-hosted server, you can quickly build applications such as multi-modal AI like voice AI agents for NPCs, live streaming, or video calls with just a few lines of code. @@ -40,34 +28,23 @@ Use this SDK to add realtime video, audio and data features to your Unity app. B ## Platform Support We officially support Unity 2022 onwards, we test on Unity 2022.3.62 and Unity 6000.3.10. - - [x] Windows - - [x] MacOS - - [x] Linux - - [x] iOS - - [x] Android - - [ ] WebGL - We plan to support all Unity platforms with this SDK. WebGL is currently supported with [client-sdk-unity-web](https://github.com/livekit/client-sdk-unity-web). - ## Installation - ### Using Git Before cloning the repo, make sure that Git LFS is installed and setup. - You can either clone the repo and import from the local Unity package files: - ```sh git clone https://github.com/livekit/client-sdk-unity.git @@ -76,7 +53,6 @@ cd client-sdk-unity ``` - Or you can import the Git url `https://github.com/livekit/client-sdk-unity.git` from the package manager. If you want to use tagged release versions, use `https://github.com/livekit/client-sdk-unity.git#vX.X.X`, for example with `#v1.3.5`. @@ -84,127 +60,72 @@ If you want to use tagged release versions, use `https://github.com/livekit/clie The package is also hosted in the OpenUPM package registry. Here is the guide on how to use the OpenUPM registry to import the package: https://openupm.com/packages/io.livekit.livekit-sdk/#modal-manualinstallation - ## Samples The repo contains these sample projects: - [Meet](https://github.com/livekit/client-sdk-unity/tree/main/Samples~/Meet) - [Agents](https://github.com/livekit/client-sdk-unity/tree/main/Samples~/Agents) - ## Local Development - ### Building LiveKit plugins locally - - For local development, initialize the Git submodule containing the Rust code for the LiveKit plugin libraries. - - There is a [helper script](https://github.com/livekit/client-sdk-unity/blob/main/Scripts~/build_ffi_locally.sh) to build the libraries locally and exchange the downloaded libraries with the local build artifacts in the correct `Runtime/Plugins` folder. - - Currently, the build script supports the following arguments: - macos - - android - - ios - - In the following options: - debug (default) - - release - - So a build command is for example: -`./Scripts~/build_ffi_locally.sh macos release` - - +`./Scripts~/build_ffi_locally.sh macos release` ### VSCode setup - - Look at the Unity-SDK.code-workspace setup for VSCode. This will use the Meet Sample as the Unity project and the Unity SDK package as two roots in a multi-root workspace and the Meet.sln as the `dotnet.defaultSolution`, enabling Rust and C# IDE support. - - ### Debugging - - For C# debugging, there is a simple attach launch option called `C# Unity`, for example in the `Meet/.vscode/launch.json`. - - For Rust / C++ debugging on MacOS, you need to install the [CodeLLDB](https://marketplace.visualstudio.com/items?itemName=vadimcn.vscode-lldb) extension. The debug attach is defined in `.vscode/launch.json`. - - 1. Build the livekit-ffi lib locally in debug mode with `./Scripts~/build_ffi_locally.sh macos debug` - 2. Start the Unity Editor - 3. Attach to the Unity Editor process (either auto or manual process picker) - 4. Start the Scene in Editor - - ### iOS - - Add dependent frameworks to your Unity project - - select `Unity-iPhone` -> `TARGETS` -> `UnityFramework` -> `General` -> `Frameworks and Libraries` -> `+` - - add the following frameworks: - - `OpenGLES.framework` `MetalKit.framework` `GLKit.framework` `MetalKit.framework` `VideoToolBox.framework` `Network.framework` - - add other linker flags to `UnityFramework`: - - `-ObjC` - - - -Since `libiPhone-lib.a` has built-in old versions of `celt` and `libvpx` (This will cause the opus and vp8/vp9 codecs to not be called correctly and cause a crash.), you need to ensure that `liblivekit_ffi.a` is linked before `libiPhone-lib.a`. - +Since `libiPhone-lib.a` has built-in old versions of `celt` and `libvpx` (This will cause the opus and vp8/vp9 codecs to not be called correctly and cause a crash.), you need to ensure that `liblivekit_ffi.a` is linked before `libiPhone-lib.a`. The package now applies an iOS post-build fix that rewrites the exported Xcode project so `libiPhone-lib.a` is moved after `liblivekit_ffi.a` in `UnityFramework -> Frameworks and Libraries`. - - It also strips the old CELT object cluster from the exported `Libraries/libiPhone-lib.a` so Xcode cannot resolve those codec symbols from Unity's archive. - - If your project disables package editor scripts or uses a custom Xcode export pipeline that overwrites `project.pbxproj` after LiveKit runs, you may still need to adjust the order manually by removing and re-adding `libiPhone-lib.a`. - - ## Examples ### Tokens @@ -260,111 +181,23 @@ ITokenSourceFixed source = new TokenSourceLiteral("wss://your.livekit.host", " await MyAuthFlow()); ``` - - -### Connect to a room +### Connecting to a room - ```cs - -using LiveKit; - - - -IEnumerator Start() - +private IEnumerator ConnectToRoom() { + var serverUrl = "< your server url >"; + var token = "< your token >"; -var room = new Room(); - -room.TrackSubscribed += TrackSubscribed; - - - -var connect = room.Connect("ws://localhost:7880", ""); - -yield return connect; - -if (!connect.IsError) - -{ - -Debug.Log("Connected to " + room.Name); - -} - -} - -``` - - - -## Asynchronous programming: coroutines, async/await, and UniTask - -The SDK exposes three interchangeable styles for awaiting asynchronous operations. Coroutines, async/await and UniTask. - -**1. Coroutines (default, no dependency)** — shown throughout this README. - -**2. async/await (no dependency)** — every operation returns an awaitable instruction (`ConnectInstruction`, `PublishTrackInstruction`, `PerformRpcInstruction`, the stream read instructions, …), so you can `await` it directly. As with coroutines, you inspect success/failure on the instruction (`IsError`) — `await` does not throw. Continuations resume on Unity's main thread. - -```cs -async void Start() -{ var room = new Room(); - var connect = room.Connect("ws://localhost:7880", "", new RoomOptions()); - await connect; - if (!connect.IsError) - Debug.Log("Connected to " + room.Name); -} -``` - -> Use `async void` only for top-level event handlers (e.g. button callbacks); its exceptions surface to Unity's log rather than to a caller. Prefer `async Task`/`async UniTaskVoid` elsewhere. - -**3. UniTask (optional)** — install [UniTask](https://github.com/Cysharp/UniTask) (`com.cysharp.unitask`). The SDK auto-detects it via the `LIVEKIT_UNITASK` scripting define and enables the `LiveKit.UniTask` assembly, which adds `CancellationToken` support, composition, and async streams. - -Cancellation (abandon-awaiter semantics — the underlying request is not cancelled on the wire): - -```cs -await room.Connect("ws://localhost:7880", "", new RoomOptions()) - .AsUniTask(cancellationToken); -``` - -Run operations in parallel. `AsUniTask` does not throw on failure (matching the -coroutine path), so keep the instructions and check `IsError` on each after the -`await` — otherwise a failed operation passes silently: - -```cs -var publishCamera = room.LocalParticipant.PublishTrack(cameraTrack, cameraOptions); -var publishMicrophone = room.LocalParticipant.PublishTrack(microphoneTrack, microphoneOptions); + var connect = room.Connect(serverUrl, token, new RoomOptions()); -await UniTask.WhenAll(publishCamera.AsUniTask(ct), publishMicrophone.AsUniTask(ct)); - -if (publishCamera.IsError || publishMicrophone.IsError) - Debug.LogError("Failed to publish one or more tracks"); -``` - -Consume an incremental stream with `await foreach`. The sequence ends at end-of-stream; if the stream ends with an error it throws a `StreamError`: - -```cs -try -{ - await foreach (var chunk in reader.ReadIncremental().AsAsyncEnumerable(ct)) - Process(chunk); -} -catch (StreamError e) -{ - Debug.LogError(e.Message); + yield return connect; } ``` -> Error-handling differs by API: awaiting an instruction (and `AsUniTask`) never throws on a -> failed operation — you inspect `IsError` after the `await`. The stream enumerable is the -> exception: `await foreach` has no post-loop point to check `IsError`, so a mid-stream failure -> surfaces by throwing `StreamError`. - - ### Publishing microphone @@ -1029,8 +862,71 @@ StartCoroutine(HandleByteStream(reader, identity)) ``` +## Asynchronous programming: coroutines, async/await, and UniTask + +The SDK exposes three interchangeable styles for awaiting asynchronous operations. Coroutines, async/await and UniTask. + +**1. Coroutines (default, no dependency)** — shown throughout this README. + +**2. async/await (no dependency)** — every operation returns an awaitable instruction (`ConnectInstruction`, `PublishTrackInstruction`, `PerformRpcInstruction`, the stream read instructions, …), so you can `await` it directly. As with coroutines, you inspect success/failure on the instruction (`IsError`) — `await` does not throw. Continuations resume on Unity's main thread. + +```cs +async void Start() +{ + var room = new Room(); + var connect = room.Connect("ws://localhost:7880", "", new RoomOptions()); + await connect; + if (!connect.IsError) + Debug.Log("Connected to " + room.Name); +} +``` + +> Use `async void` only for top-level event handlers (e.g. button callbacks); its exceptions surface to Unity's log rather than to a caller. Prefer `async Task`/`async UniTaskVoid` elsewhere. + +**3. UniTask (optional)** — install [UniTask](https://github.com/Cysharp/UniTask) (`com.cysharp.unitask`). The SDK auto-detects it via the `LIVEKIT_UNITASK` scripting define and enables the `LiveKit.UniTask` assembly, which adds `CancellationToken` support, composition, and async streams. + +Cancellation (abandon-awaiter semantics — the underlying request is not cancelled on the wire): + +```cs +await room.Connect("ws://localhost:7880", "", new RoomOptions()) + .AsUniTask(cancellationToken); +``` + +Run operations in parallel. `AsUniTask` does not throw on failure (matching the +coroutine path), so keep the instructions and check `IsError` on each after the +`await` — otherwise a failed operation passes silently: + +```cs +var publishCamera = room.LocalParticipant.PublishTrack(cameraTrack, cameraOptions); +var publishMicrophone = room.LocalParticipant.PublishTrack(microphoneTrack, microphoneOptions); + +await UniTask.WhenAll(publishCamera.AsUniTask(ct), publishMicrophone.AsUniTask(ct)); + +if (publishCamera.IsError || publishMicrophone.IsError) + Debug.LogError("Failed to publish one or more tracks"); +``` + +Consume an incremental stream with `await foreach`. The sequence ends at end-of-stream; if the stream ends with an error it throws a `StreamError`: + +```cs +try +{ + await foreach (var chunk in reader.ReadIncremental().AsAsyncEnumerable(ct)) + Process(chunk); +} +catch (StreamError e) +{ + Debug.LogError(e.Message); +} +``` + +> Error-handling differs by API: awaiting an instruction (and `AsUniTask`) never throws on a +> failed operation — you inspect `IsError` after the `await`. The stream enumerable is the +> exception: `await foreach` has no post-loop point to check `IsError`, so a mid-stream failure +> surfaces by throwing `StreamError`. + ## Verbose Logging From 6449263d5925fdff176cd846b24844c79e42f4cf Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Wed, 15 Jul 2026 09:07:03 +0200 Subject: [PATCH 11/19] Aligned with docs --- README.md | 231 ++++++++---------------------------------------------- 1 file changed, 34 insertions(+), 197 deletions(-) diff --git a/README.md b/README.md index e08cd2d4..15c2d738 100644 --- a/README.md +++ b/README.md @@ -60,12 +60,6 @@ If you want to use tagged release versions, use `https://github.com/livekit/clie The package is also hosted in the OpenUPM package registry. Here is the guide on how to use the OpenUPM registry to import the package: https://openupm.com/packages/io.livekit.livekit-sdk/#modal-manualinstallation -## Samples - -The repo contains these sample projects: -- [Meet](https://github.com/livekit/client-sdk-unity/tree/main/Samples~/Meet) -- [Agents](https://github.com/livekit/client-sdk-unity/tree/main/Samples~/Agents) - ## Local Development ### Building LiveKit plugins locally @@ -128,6 +122,12 @@ If your project disables package editor scripts or uses a custom Xcode export pi ## Examples +The repo contains these sample projects: +- [Meet](https://github.com/livekit/client-sdk-unity/tree/main/Samples~/Meet) +- [Agents](https://github.com/livekit/client-sdk-unity/tree/main/Samples~/Agents) + +Most of the following functionalities and code snippets can be found in the samples in a similar form to try out. + ### Tokens You need a token to join a LiveKit room as a participant. Read more about tokens here: https://docs.livekit.io/frontends/reference/tokens-grants/ @@ -181,9 +181,7 @@ ITokenSourceFixed source = new TokenSourceLiteral("wss://your.livekit.host", " await MyAuthFlow()); ``` - ### Connecting to a room - ```cs private IEnumerator ConnectToRoom() @@ -199,7 +197,6 @@ private IEnumerator ConnectToRoom() ``` ### Publishing microphone - ```cs @@ -246,12 +243,8 @@ rtcSource.Start(); ``` - - ### Publishing a texture (e.g Unity Camera) - - ```cs // publish a WebCamera @@ -326,12 +319,8 @@ StartCoroutine(source.Update()); ``` - - ### Receiving tracks - - ```cs IEnumerator Start() @@ -400,12 +389,8 @@ var stream = new AudioStream(audioTrack, source); ``` - - ### RPC - - Perform your own predefined method calls from one participant to another. @@ -416,12 +401,8 @@ This feature is especially powerful when used with [Agents](https://docs.livekit The following is a brief overview but [more detail is available in the documentation](https://docs.livekit.io/home/client/data/rpc). - - #### Registering an RPC method - - The participant who implements the method and will receive its calls must first register support. Your method handler will be an async callback that receives an `RpcInvocationData` object: @@ -448,20 +429,12 @@ room.LocalParticipant.RegisterRpcMethod("greet", HandleGreeting); ``` - - In addition to the payload, `RpcInvocationData` also contains `responseTimeout`, which informs you the maximum time available to return a response. If you are unable to respond in time, the call will result in an error on the caller's side. - - #### Performing an RPC request - - The caller may initiate an RPC call using coroutines: - - ```cs IEnumerator PerformRpcCoroutine() @@ -512,192 +485,74 @@ StartCoroutine(PerformRpcCoroutine()); ``` - - You may find it useful to adjust the `ResponseTimeout` parameter, which indicates the amount of time you will wait for a response. We recommend keeping this value as low as possible while still satisfying the constraints of your application. - - #### Errors - - LiveKit is a dynamic realtime environment and RPC calls can fail for various reasons. - - You may throw errors of the type `RpcError` with a string `message` in an RPC method handler and they will be received on the caller's side with the message intact. Other errors will not be transmitted and will instead arrive to the caller as `1500` ("Application Error"). Other built-in errors are detailed in the [docs](https://docs.livekit.io/home/client/data/rpc/#errors). - - ### Sending text - - Use text streams to send any amount of text between participants. - - #### Sending text all at once - - ```cs - -IEnumerator PerformSendText() - +IEnumerator SendText() { - -var text = "Lorem ipsum dolor sit amet..."; - -var sendTextCall = room.LocalParticipant.SendText(text, "some-topic"); - -yield return sendTextCall; - - - -Debug.Log($"Sent text with stream ID {sendTextCall.Info.Id}"); - + var text = "Lorem ipsum dolor sit amet..."; + var sendTextInstruction = room.LocalParticipant.SendText("Hello from Unity", "Chat"); + yield return sendTextInstruction; } - ``` - - #### Streaming text incrementally - - ```cs - -IEnumerator PerformStreamText() - -{ - -var streamTextCall = room.LocalParticipant.StreamText("my-topic"); - -yield return streamTextCall; - - - -var writer = streamTextCall.Writer; - -Debug.Log($"Opened text stream with ID: {writer.Info.Id}"); - - - -// In a real app, you would generate this text asynchronously / incrementally as well - -var textChunks = new[] { "Lorem ", "ipsum ", "dolor ", "sit ", "amet..." }; - -foreach (var chunk in textChunks) - +IEnumerator StreamText(Room room) { - -yield return writer.Write(chunk); - -} - - - -// The stream must be explicitly closed when done - -yield return writer.Close(); - - - -Debug.Log($"Closed text stream with ID: {writer.Info.Id}"); - + var streamWriter = room.LocalParticipant.StreamText("Chat"); + yield return streamWriter; + string[] textChunks = {"Lorem ", "ipsum ", "dolor ", "sit ", "amet..."}; + foreach (var textChunk in textChunks) + { + var instruction = streamWriter.Writer.Write(textChunk); + yield return instruction; + } + yield return streamWriter.Close(); } - ``` - - #### Handling incoming streams - - ```cs - -IEnumerator HandleTextStream(TextStreamReader reader, string participantIdentity) - +void OnRoomConnected(Room room) { - -var info = reader.Info; - -Debug.Log($@" - -Text stream received from {participantIdentity} - -Topic: {info.Topic} - -Timestamp: {info.Timestamp} - -ID: {info.Id} - -Size: {info.TotalLength} (only available if the stream was sent with `SendText`) - -"); - - - -// Option 1: Process the stream incrementally - -var readIncremental = reader.ReadIncremental(); - -while (true) - -{ - -readIncremental.Reset(); - -yield return readIncremental; - -if (readIncremental.IsEos) break; - -Debug.Log($"Next chunk: {readIncremental.Text}"); - + room.RegisterTextStreamHandler("Chat", (reader, identity) => {StartCoroutine(OnTextStream(reader, identity));}); } - - -// Option 2: Get the entire text after the stream completes - -var readAllCall = reader.ReadAll(); - -yield return readAllCall; - -Debug.Log($"Received text: {readAllCall.Text}") - +IEnumerator OnTextStream(TextStreamReader reader, string identity) +{ + var readIncremental = reader.ReadIncremental(); + while (true) + { + readIncremental.Reset(); + yield return readIncremental; + if (readIncremental.IsEos) + break; + Debug.Log(readIncremental.Text); + } } - - - -// Register the topic before connecting to the room - -room.RegisterTextStreamHandler("my-topic", (reader, identity) => - -StartCoroutine(HandleTextStream(reader, identity)) - -); - ``` - - ### Sending files & bytes - - Use byte streams to send files, images, or any other kind of data between participants. - - #### Sending files - - ```cs IEnumerator PerformSendFile() @@ -718,12 +573,8 @@ Debug.Log($"Sent file with stream ID: {sendFileCall.Info.Id}"); ``` - - #### Streaming bytes - - ```cs IEnumerator PerformStreamBytes() @@ -776,12 +627,8 @@ Debug.Log($"Closed byte stream with ID: {writer.Info.Id}"); ``` - - #### Handling incoming streams - - ```cs IEnumerator HandleByteStream(ByteStreamReader reader, string participantIdentity) @@ -925,23 +772,13 @@ catch (StreamError e) > exception: `await foreach` has no post-loop point to check `IsError`, so a mid-stream failure > surfaces by throwing `StreamError`. - - ## Verbose Logging - - To enable verbose logging, define the `LK_VERBOSE` symbol: - - 1. Navigate to Project Settings → Player - 2. Select your platform tab (e.g., Mac, iOS, Android). - -3. Under Other Settings → Scripting Define Symbols, add `LK_VERBOSE`. - - +3. Under Other Settings → Scripting Define Symbols, add `LK_VERBOSE`. From 1a4122f89d3ccff71389e5b74fc0ad8003208826 Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Wed, 15 Jul 2026 13:08:58 +0200 Subject: [PATCH 12/19] Improved data and byte stream examples --- README.md | 200 ++++++++++++++++++------------------------------------ 1 file changed, 67 insertions(+), 133 deletions(-) diff --git a/README.md b/README.md index 15c2d738..8b616958 100644 --- a/README.md +++ b/README.md @@ -518,10 +518,12 @@ IEnumerator StreamText(Room room) string[] textChunks = {"Lorem ", "ipsum ", "dolor ", "sit ", "amet..."}; foreach (var textChunk in textChunks) { + Debug.Log($"Sending {textChunk}"); var instruction = streamWriter.Writer.Write(textChunk); yield return instruction; } - yield return streamWriter.Close(); + + yield return streamWriter.Writer.Close(); } ``` @@ -530,11 +532,12 @@ IEnumerator StreamText(Room room) ```cs void OnRoomConnected(Room room) { - room.RegisterTextStreamHandler("Chat", (reader, identity) => {StartCoroutine(OnTextStream(reader, identity));}); + room.RegisterTextStreamHandler("Chat", (reader, identity) => StartCoroutine(OnTextStream(reader, identity))); } IEnumerator OnTextStream(TextStreamReader reader, string identity) { + // Option 1: Process the stream incrementally var readIncremental = reader.ReadIncremental(); while (true) { @@ -544,6 +547,11 @@ IEnumerator OnTextStream(TextStreamReader reader, string identity) break; Debug.Log(readIncremental.Text); } + + // Option 2: Get the entire text after the stream completes + var readAllCall = reader.ReadAll(); + yield return readAllCall; + Debug.Log($"Received text: {readAllCall.Text}"); } ``` @@ -554,159 +562,85 @@ Use byte streams to send files, images, or any other kind of data between partic #### Sending files ```cs - -IEnumerator PerformSendFile() - +IEnumerator SendFile() { - -var filePath = "path/to/file.jpg"; - -var sendFileCall = room.LocalParticipant.SendFile(filePath, "some-topic"); - -yield return sendFileCall; - - - -Debug.Log($"Sent file with stream ID: {sendFileCall.Info.Id}"); - + var filePath = "path/to/file.jpg"; + Debug.Log($"Sending file {filePath}"); + var sendFileCall = room.LocalParticipant.SendFile(filePath, "my-topic"); + yield return sendFileCall; } - ``` #### Streaming bytes ```cs - -IEnumerator PerformStreamBytes() - +IEnumerator StreamBytes() { + var streamBytesCall = room.LocalParticipant.StreamBytes("my-topic"); + yield return streamBytesCall; -var streamBytesCall = room.LocalParticipant.StreamBytes("my-topic"); - -yield return streamBytesCall; - - - -var writer = streamBytesCall.Writer; - -Debug.Log($"Opened byte stream with ID: {writer.Info.Id}"); - - - -// Example sending arbitrary binary data - -// For sending files, use `SendFile` instead - -var dataChunks = new[] { - -new byte[] { 0x00, 0x01 }, + var writer = streamBytesCall.Writer; + Debug.Log($"Opened byte stream with ID: {writer.Info.Id}"); -new byte[] { 0x03, 0x04 } - -}; - -foreach (var chunk in dataChunks) - -{ - -yield return writer.Write(chunk); - -} - - - -// The stream must be explicitly closed when done - -yield return writer.Close(); - - + var dataChunks = new[] + { + new byte[] { 0x00, 0x01 }, + new byte[] { 0x02, 0x03 } + }; -Debug.Log($"Closed byte stream with ID: {writer.Info.Id}"); + foreach (var chunk in dataChunks) + { + yield return writer.Write(chunk); + } + yield return writer.Close(); } - ``` #### Handling incoming streams ```cs - -IEnumerator HandleByteStream(ByteStreamReader reader, string participantIdentity) - +IEnumerator HandleByteStream(ByteStreamReader reader, string participantIdentity) { - -var info = reader.Info; - - - -// Option 1: Process the stream incrementally - -var readIncremental = reader.ReadIncremental(); - -while (true) - -{ - -readIncremental.Reset(); - -yield return readIncremental; - -if (readIncremental.IsEos) break; - -Debug.Log($"Next chunk: {readIncremental.Bytes}"); - + var info = reader.Info; + + // Option 1: Process the stream incrementally + var readIncremental = reader.ReadIncremental(); + while (true) + { + readIncremental.Reset(); + yield return readIncremental; + if (readIncremental.IsEos) break; + foreach (var dataByte in readIncremental.Bytes) + Debug.Log($"Received {dataByte}"); + } + + // Option 2: Get the entire file after the stream completes + var readAllCall = reader.ReadAll(); + yield return readAllCall; + var data = readAllCall.Bytes; + foreach (var dataByte in data) + Debug.Log($"Received {dataByte}"); + + // Option 3: Write the stream to a local file on disk as it arrives + var writeToFileCall = reader.WriteToFile(); + yield return writeToFileCall; + var path = writeToFileCall.FilePath; + Debug.Log($"Wrote to file: {path}"); + + Debug.Log($@" + Byte stream received from {participantIdentity} + Topic: {info.Topic} + Timestamp: {info.Timestamp} + ID: {info.Id} + Size: {info.TotalLength} (only available if the stream was sent with `SendFile`) + "); } - - -// Option 2: Get the entire file after the stream completes - -var readAllCall = reader.ReadAll(); - -yield return readAllCall; - -var data = readAllCall.Bytes; - - - -// Option 3: Write the stream to a local file on disk as it arrives - -var writeToFileCall = reader.WriteToFile(); - -yield return writeToFileCall; - -var path = writeToFileCall.FilePath; - -Debug.Log($"Wrote to file: {path}"); - - - -Debug.Log($@" - -Byte stream received from {participantIdentity} - -Topic: {info.Topic} - -Timestamp: {info.Timestamp} - -ID: {info.Id} - -Size: {info.TotalLength} (only available if the stream was sent with `SendFile`) - -"); - +void OnRoomConnected(Room room) +{ + room.RegisterByteStreamHandler("my-topic", (reader, identity) => StartCoroutine(HandleByteStream(reader, identity))); } - - - -// Register the topic after connection to the room - -room.RegisterByteStreamHandler("my-topic", (reader, identity) => - -StartCoroutine(HandleByteStream(reader, identity)) - -); - ``` ## Asynchronous programming: coroutines, async/await, and UniTask From b82640e0c7618ff78c688caf1f69ed625111d64e Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Wed, 15 Jul 2026 13:09:18 +0200 Subject: [PATCH 13/19] Agent targeting IL2CPP and ARM64 --- Samples~/Agents/ProjectSettings/ProjectSettings.asset | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/Samples~/Agents/ProjectSettings/ProjectSettings.asset b/Samples~/Agents/ProjectSettings/ProjectSettings.asset index f28f8e67..91ba1f5f 100644 --- a/Samples~/Agents/ProjectSettings/ProjectSettings.asset +++ b/Samples~/Agents/ProjectSettings/ProjectSettings.asset @@ -265,7 +265,7 @@ PlayerSettings: useCustomGradlePropertiesTemplate: 0 useCustomGradleSettingsTemplate: 0 useCustomProguardFile: 0 - AndroidTargetArchitectures: 1 + AndroidTargetArchitectures: 2 AndroidTargetDevices: 0 AndroidSplashScreenScale: 0 androidSplashScreen: {fileID: 0} @@ -763,7 +763,8 @@ PlayerSettings: scriptingDefineSymbols: {} additionalCompilerArguments: {} platformArchitecture: {} - scriptingBackend: {} + scriptingBackend: + Android: 1 il2cppCompilerConfiguration: {} il2cppCodeGeneration: {} managedStrippingLevel: From 9fa8964df9ce3c480a76c59fea2655ab617172a1 Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Wed, 15 Jul 2026 14:11:20 +0200 Subject: [PATCH 14/19] Rpc method done --- README.md | 93 +++++++++++++++++-------------------------------------- 1 file changed, 28 insertions(+), 65 deletions(-) diff --git a/README.md b/README.md index 8b616958..33f197af 100644 --- a/README.md +++ b/README.md @@ -405,28 +405,18 @@ The following is a brief overview but [more detail is available in the documenta The participant who implements the method and will receive its calls must first register support. Your method handler will be an async callback that receives an `RpcInvocationData` object: - ```cs - -// Define your method handler - -async Task HandleGreeting(RpcInvocationData data) - +void OnRoomConnected(Room room) { - -Debug.Log($"Received greeting from {data.CallerIdentity}: {data.Payload}"); - -return $"Hello, {data.CallerIdentity}!"; - + room.LocalParticipant.RegisterRpcMethod("greet", HandleGreeting); } - - -// Register the method after connection to the room - -room.LocalParticipant.RegisterRpcMethod("greet", HandleGreeting); - +private async Task HandleGreeting(RpcInvocationData data) +{ + Debug.Log($"Received greeting from {data.CallerIdentity}: {data.Payload}"); + return $"Hello, {data.CallerIdentity}!"; +} ``` In addition to the payload, `RpcInvocationData` also contains `responseTimeout`, which informs you the maximum time available to return a response. If you are unable to respond in time, the call will result in an error on the caller's side. @@ -436,53 +426,26 @@ In addition to the payload, `RpcInvocationData` also contains `responseTimeout`, The caller may initiate an RPC call using coroutines: ```cs - -IEnumerator PerformRpcCoroutine() - -{ - -var rpcCall = room.LocalParticipant.PerformRpc(new PerformRpcParams - -{ - -DestinationIdentity = "recipient-identity", - -Method = "greet", - -Payload = "Hello from RPC!" - -}); - - - -yield return rpcCall; - - - -if (rpcCall.IsError) - -{ - -Debug.Log($"RPC call failed: {rpcCall.Error}"); - -} - -else - +IEnumerator PerformRpcCoroutine(Room room) { + var rpcCall = room.LocalParticipant.PerformRpc(new PerformRpcParams + { + DestinationIdentity = "recipient-identity", + Method = "greet", + Payload = "Hello from RPC!" + }); -Debug.Log($"RPC response: {rpcCall.Payload}"); - -} + yield return rpcCall; + if (rpcCall.IsError) + { + Debug.Log($"RPC call failed: {rpcCall.Error}"); + } + else + { + Debug.Log($"RPC response: {rpcCall.Payload}"); + } } - - - -// Start the coroutine from another MonoBehaviour method - -StartCoroutine(PerformRpcCoroutine()); - ``` You may find it useful to adjust the `ResponseTimeout` parameter, which indicates the amount of time you will wait for a response. We recommend keeping this value as low as possible while still satisfying the constraints of your application. @@ -600,6 +563,11 @@ IEnumerator StreamBytes() #### Handling incoming streams ```cs +void OnRoomConnected(Room room) +{ + room.RegisterByteStreamHandler("my-topic", (reader, identity) => StartCoroutine(HandleByteStream(reader, identity))); +} + IEnumerator HandleByteStream(ByteStreamReader reader, string participantIdentity) { var info = reader.Info; @@ -636,11 +604,6 @@ IEnumerator HandleByteStream(ByteStreamReader reader, string participantIdentity Size: {info.TotalLength} (only available if the stream was sent with `SendFile`) "); } - -void OnRoomConnected(Room room) -{ - room.RegisterByteStreamHandler("my-topic", (reader, identity) => StartCoroutine(HandleByteStream(reader, identity))); -} ``` ## Asynchronous programming: coroutines, async/await, and UniTask From 5d3814676d1f7a47be9052ddae8a100de5f1d36e Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Wed, 15 Jul 2026 14:31:50 +0200 Subject: [PATCH 15/19] Publishing camera track --- README.md | 103 ++++++++++++++++-------------------------------------- 1 file changed, 31 insertions(+), 72 deletions(-) diff --git a/README.md b/README.md index 33f197af..df8f175f 100644 --- a/README.md +++ b/README.md @@ -237,86 +237,50 @@ Debug.Log("Track published!"); } - - rtcSource.Start(); - ``` ### Publishing a texture (e.g Unity Camera) ```cs +IEnumerator PublishCamera(Room room) +{ + // Option 1: publish a WebCamera + // var source = new TextureVideoSource(webCamTexture); -// publish a WebCamera - -//var source = new TextureVideoSource(webCamTexture); - - - -// Publish the entire screen - -//var source = new ScreenVideoSource(); - - - -// Publishing a Unity Camera - -//Camera.main.enabled = true; - -//var source = new CameraVideoSource(Camera.main); - - - -var rt = new UnityEngine.RenderTexture(1920, 1080, 24, RenderTextureFormat.ARGB32); - -rt.Create(); - -Camera.main.targetTexture = rt; - -var source = new TextureVideoSource(rt); - -var track = LocalVideoTrack.CreateVideoTrack("my-video-track", source, room); - - - -var options = new TrackPublishOptions(); - -options.VideoCodec = VideoCodec.Vp8; - -var videoCoding = new VideoEncoding(); - -videoCoding.MaxBitrate = 512000; - -videoCoding.MaxFramerate = frameRate; - -options.VideoEncoding = videoCoding; - -options.Simulcast = true; + // Option 2: publish a screen share + // var source = new ScreenVideoSource(); -options.Source = TrackSource.SourceCamera; - - - -var publish = room.LocalParticipant.PublishTrack(track, options); - -yield return publish; + // Option 3: publishing a Unity Camera + var source = new CameraVideoSource(Camera.main); + + var track = LocalVideoTrack.CreateVideoTrack("my-video-track", source, room); + + var videoCoding = new VideoEncoding + { + MaxBitrate = 512000, + MaxFramerate = frameRate + }; - + var options = new TrackPublishOptions + { + VideoCodec = VideoCodec.Vp8, + VideoEncoding = videoCoding, + Simulcast = true, + Source = TrackSource.SourceCamera + }; -if (!publish.IsError) + var publish = room.LocalParticipant.PublishTrack(track, options); + yield return publish; -{ - -Debug.Log("Track published!"); + if (!publish.IsError) + { + Debug.Log("Track published!"); + } + source.Start(); + StartCoroutine(source.Update()); } - - - -source.Start(); - -StartCoroutine(source.Update()); - ``` ### Receiving tracks @@ -393,19 +357,14 @@ var stream = new AudioStream(audioTrack, source); Perform your own predefined method calls from one participant to another. - - This feature is especially powerful when used with [Agents](https://docs.livekit.io/agents), for instance to forward LLM function calls to your client application. - - The following is a brief overview but [more detail is available in the documentation](https://docs.livekit.io/home/client/data/rpc). #### Registering an RPC method The participant who implements the method and will receive its calls must first register support. Your method handler will be an async callback that receives an `RpcInvocationData` object: - ```cs void OnRoomConnected(Room room) { From c3ddf80f8413f449b401c599f132b51910fcbd26 Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Wed, 15 Jul 2026 15:36:52 +0200 Subject: [PATCH 16/19] Split up audio section --- README.md | 204 ++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 128 insertions(+), 76 deletions(-) diff --git a/README.md b/README.md index df8f175f..2c4387c6 100644 --- a/README.md +++ b/README.md @@ -184,7 +184,7 @@ ITokenSourceFixed source = new TokenSourceLiteral("wss://your.livekit.host", "(); + var stream = new VideoStream(videoTrack); + stream.TextureReceived += (tex) => + { + rawImage.texture = tex; + }; + + StartCoroutine(stream.Update()); + } +} +``` -IEnumerator Start() +### Audio -{ +There are two options to handle audio input and output, via "Unity Audio" or via "Platform Audio". -var room = new Room(); +#### Unity Audio -room.TrackSubscribed += TrackSubscribed; +Using Unity Audio, the Unity Microphone and AudioSource APIs are used to pipe the audio frames. Use Unity Audio if your game needs to: +- read the audio frames, e.g. for lip syncing +- manipulate the audio frames - +On mobile platforms, the WebRTC audio is handled within the audio session owned by Unity, which reduces complexity. -var connect = room.Connect("ws://localhost:7880", ""); +#### Unity Audio Input -yield return connect; +```cs +IEnumerator PublishLocalMicrophoneUnity(Room room) +{ + Debug.Log("Publishing microphone using Unity Audio"); + GameObject microphoneObject = new GameObject("my-audio-source"); + var rtcSource = new MicrophoneSource(Microphone.devices[0], microphoneObject); + var track = LocalAudioTrack.CreateAudioTrack("my-audio-track", rtcSource, room); - + var options = new TrackPublishOptions(); + options.AudioEncoding = new AudioEncoding(); + options.AudioEncoding.MaxBitrate = 64000; + options.Source = TrackSource.SourceMicrophone; -// .... + var publish = room.LocalParticipant.PublishTrack(track, options); + yield return publish; -} + if (!publish.IsError) + { + Debug.Log("Track published!"); + } - + rtcSource.Start(); +} +``` -void TrackSubscribed(IRemoteTrack track, RemoteTrackPublication publication, RemoteParticipant participant) +#### Unity Audio Output +```cs +void TrackSubscribed(IRemoteTrack track, RemoteTrackPublication publication, RemoteParticipant participant) { + if (track is RemoteAudioTrack audioTrack) + { + GameObject audioOutputObject = new GameObject(audioTrack.Sid); + var source = audioOutputObject.AddComponent(); + var stream = new AudioStream(audioTrack, source); + } +} +``` -if (track is RemoteVideoTrack videoTrack) +#### Platform Audio -{ +With Platform Audio, the audio input and output are managed by the native ADM of WebRTC. This unlocks echo cancellation, noise suppression, auto gain control and hardware processing if available. -var rawImage = GetComponent(); +There are some known issues with Platform Audio, that we are working on resolving: +- On iOS, disposing of Platform Audio object stops Unity audio output +- On iOS and Unity 6, backgrounding the app breaks Platform Audio +- On MacOS with bluetooth headset, unmuting can break audio output -var stream = new VideoStream(videoTrack); +#### Initialize Platform Audio -stream.TextureReceived += (tex) => +Make sure to initialize Platform Audio before connecting to a call. +```cs +void InitializePlatformAudio() { + try + { + var platformAudio = new PlatformAudio(); + Debug.Log($"PlatformAudio initialized: {platformAudio.RecordingDeviceCount} mics, " + + $"{platformAudio.PlayoutDeviceCount} speakers"); -rawImage.texture = tex; - -}; + var (recording, playout) = platformAudio.GetDevices(); + Debug.Log("Recording devices:"); + foreach (var device in recording) + Debug.Log($" [{device.Index}] {device.Name}"); -StartCoroutine(stream.Update()); + Debug.Log("Playout devices:"); + foreach (var device in playout) + Debug.Log($" [{device.Index}] {device.Name}"); -// The video data is displayed on the rawImage + if (platformAudio.RecordingDeviceCount > 0) + platformAudio.SetRecordingDevice(0); + if (platformAudio.PlayoutDeviceCount > 0) + platformAudio.SetPlayoutDevice(0); + Debug.Log($"PlatformAudio ready. AEC={echoCancellation}, NS={noiseSuppression}, AGC={autoGainControl}, HW={preferHardwareProcessing}"); + } + catch (System.Exception e) + { + Debug.LogError($"Failed to initialize PlatformAudio, falling back to Unity audio: {e.Message}"); + usePlatformAudio = false; + platformAudio = null; + } } +``` -else if (track is RemoteAudioTrack audioTrack) +#### Platform Audio Input +```cs +IEnumerator PublishLocalMicrophonePlatform(PlatformAudio platformAudio, Room room) { + if (platformAudio != null) + { + yield return platformAudio.StartRecording(); + } -GameObject audObject = new GameObject(audioTrack.Sid); + var audioOptions = new AudioProcessingOptions + { + EchoCancellation = echoCancellation, + NoiseSuppression = noiseSuppression, + AutoGainControl = autoGainControl, + PreferHardware = preferHardwareProcessing + }; -var source = audObject.AddComponent(); + var platformAudioSource = new PlatformAudioSource(platformAudio, audioOptions); + var localAudioTrack = LocalAudioTrack.CreateAudioTrack(LocalAudioTrackName, platformAudioSource, room); -var stream = new AudioStream(audioTrack, source); + var options = new TrackPublishOptions + { + AudioEncoding = new AudioEncoding { MaxBitrate = 64000 }, + Source = TrackSource.SourceMicrophone + }; -// Audio is being played on the source .. + var publish = room.LocalParticipant.PublishTrack(localAudioTrack, options); + yield return publish; -} + if (publish.IsError) + { + Debug.LogError("Failed to publish microphone track"); + platformAudioSource?.Dispose(); + yield break; + } + Debug.Log("Microphone published via PlatformAudio (AEC enabled)"); } - ``` +#### Platform Audio Output + +Using Platform Audio, for audio output of subscribed remote audio tracks you don't need any Unity handling. + ### RPC Perform your own predefined method calls from one participant to another. @@ -371,7 +423,7 @@ void OnRoomConnected(Room room) room.LocalParticipant.RegisterRpcMethod("greet", HandleGreeting); } -private async Task HandleGreeting(RpcInvocationData data) +async Task HandleGreeting(RpcInvocationData data) { Debug.Log($"Received greeting from {data.CallerIdentity}: {data.Payload}"); return $"Hello, {data.CallerIdentity}!"; From d6a143a28d595733d71dac8156fbd6cac94aaade Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Wed, 15 Jul 2026 17:04:35 +0200 Subject: [PATCH 17/19] Updating to TokenSource changes --- README.md | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 2c4387c6..fd09c77f 100644 --- a/README.md +++ b/README.md @@ -147,17 +147,19 @@ For production. Point to your own token endpoint URL and add any required authen Add a `TokenSourceComponent` to a GameObject, assign your `TokenSourceComponentConfig` asset, then fetch connection details before connecting: +`FetchConnectionDetails` returns a `TaskYieldInstruction`, so you can `yield return` it from a coroutine, `await` it, or bridge it with `.AsUniTask()`. Inspect the result on the instruction itself — `IsError` / `Exception` / `Result` — rather than on a `Task`: + ```cs -var connectionDetailsTask = _tokenSourceComponent.FetchConnectionDetails(); -yield return new WaitUntil(() => connectionDetailsTask.IsCompleted); +var fetch = _tokenSourceComponent.FetchConnectionDetails(); +yield return fetch; -if (connectionDetailsTask.IsFaulted) +if (fetch.IsError) { - Debug.LogError($"Failed to fetch connection details: {connectionDetailsTask.Exception?.InnerException?.Message}"); + Debug.LogError($"Failed to fetch connection details: {fetch.Exception?.Message}"); yield break; } -var details = connectionDetailsTask.Result; +var details = fetch.Result; _room = new Room(); var connect = _room.Connect(details.ServerUrl, details.ParticipantToken, new RoomOptions()); ``` @@ -165,20 +167,30 @@ var connect = _room.Connect(details.ServerUrl, details.ParticipantToken, new Roo Per-call overrides (e.g. dynamic room or participant names) can be passed via `TokenSourceFetchOptions`; any field set there wins over the asset, and unset fields fall back to the config: ```cs -var task = _tokenSourceComponent.FetchConnectionDetails(new TokenSourceFetchOptions +var fetch = _tokenSourceComponent.FetchConnectionDetails(new TokenSourceFetchOptions { RoomName = "lobby-" + System.Guid.NewGuid(), ParticipantName = playerName, }); ``` -To skip the ScriptableObject entirely, instantiate a token source directly: +To skip the ScriptableObject entirely, instantiate a token source directly. Each returns the same `TaskYieldInstruction` from `FetchConnectionDetails`, so it can be yielded, awaited, or `.AsUniTask()`-bridged just like the component: ```cs +// Fixed sources take no per-call options: ITokenSourceFixed source = new TokenSourceLiteral("wss://your.livekit.host", ""); -// or: new TokenSourceSandbox(""); -// or: new TokenSourceEndpoint("https://your.token-server/api/token", headers); // or: new TokenSourceCustom(async () => await MyAuthFlow()); + +var fetch = source.FetchConnectionDetails(); +yield return fetch; +var details = fetch.Result; + +// Configurable sources accept TokenSourceFetchOptions per call: +ITokenSourceConfigurable configurable = new TokenSourceSandbox(""); +// or: new TokenSourceEndpoint("https://your.token-server/api/token", headers); + +var configurableFetch = configurable.FetchConnectionDetails(new TokenSourceFetchOptions { RoomName = "lobby" }); +yield return configurableFetch; ``` ### Connecting to a room From c9d4a7fd897e47bd9be2e2478888f0e3c1ecd82c Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Thu, 16 Jul 2026 10:51:31 +0200 Subject: [PATCH 18/19] Adding error log to file not found --- README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README.md b/README.md index fd09c77f..1237c671 100644 --- a/README.md +++ b/README.md @@ -554,6 +554,11 @@ IEnumerator SendFile() Debug.Log($"Sending file {filePath}"); var sendFileCall = room.LocalParticipant.SendFile(filePath, "my-topic"); yield return sendFileCall; + + if (sendFileCall.IsError) + { + Debug.LogError("File not found"); + } } ``` From 277529a5984e06d7aea518dfc208a55a2d448918 Mon Sep 17 00:00:00 2001 From: Max Heimbrock <43608204+MaxHeimbrock@users.noreply.github.com> Date: Fri, 17 Jul 2026 17:09:48 +0200 Subject: [PATCH 19/19] Add real youtube tutorial link --- README.md | 2 +- Samples~/Agents/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 1237c671..5e9ca9f1 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ Use this SDK to add realtime video, audio and data features to your Unity app. B Screenshot of the LiveKit Unity SDK Youtube tutorial -[Youtube Agents NPC Tutorial](https://www.youtube.com/@livekit_io) +[Youtube Agents NPC Tutorial](https://www.youtube.com/watch?v=cLreGfOKzN8) ## Platform Support diff --git a/Samples~/Agents/README.md b/Samples~/Agents/README.md index c63ac74a..e021e48a 100644 --- a/Samples~/Agents/README.md +++ b/Samples~/Agents/README.md @@ -6,7 +6,7 @@ Connect to a voice AI agent. Screenshot of the LiveKit Unity SDK Youtube tutorial -[Youtube Agents NPC Tutorial](https://www.youtube.com/@livekit_io) +[Youtube Agents NPC Tutorial](https://www.youtube.com/watch?v=cLreGfOKzN8) ## Getting started