Sxian/clt 2919/integrate platform audio

[xianshijing-lk]

Summary

Adds PlatformAudio support to the Python SDK, enabling microphone capture via WebRTC's Audio Device Module (ADM) with built-in voice processing.

Changes

New: livekit-rtc/livekit/rtc/platform_audio.py

• PlatformAudio - Main class for ADM access
  • recording_devices() / playout_devices() - Device enumeration
  • set_recording_device() / set_playout_device() - Device selection by GUID
  • create_audio_source() - Create audio source with processing options
• PlatformAudioSource - Audio source backed by ADM (no capture_frame() needed)
• PlatformAudioOptions - Configure AEC, NS, AGC, prefer_hardware
• AudioDeviceInfo - Device info (index, name, id/GUID)

Modified: livekit-rtc/livekit/rtc/track.py

• LocalAudioTrack.create_audio_track() now accepts both AudioSource and PlatformAudioSource

Modified: livekit-rtc/livekit/rtc/init.py

• Export: PlatformAudio, PlatformAudioSource, PlatformAudioOptions, PlatformAudioError, AudioDeviceInfo

Modified: livekit-rtc/livekit/rtc/room.py

• Commented out ready_for_room_event (not yet in released livekit-ffi)

Modified: examples/basic_room.py

• Complete rewrite demonstrating both audio modes:
  • --platform-audio - Use ADM with voice processing (recommended)
  • --file WAV_PATH - Publish WAV file via synthetic mode
  • --list-devices - List available audio devices
  • --mic-id / --speaker-id - Select specific devices
  • --room - Room name
• Demonstrates mixing both modes (microphone + file simultaneously)

New: examples/README.md

• Documentation for all examples
• Detailed explanation of PlatformAudio vs Synthetic mode

PlatformAudio vs Synthetic Mode
┌───────────────────────────────┬───────────────┬───────────────────────────────────────┐
│ Feature │ PlatformAudio │ Synthetic │
├───────────────────────────────┼───────────────┼───────────────────────────────────────┤
│ Voice processing (AEC/NS/AGC) │ Built-in │ Manual │
├───────────────────────────────┼───────────────┼───────────────────────────────────────┤
│ Raw frame access │ No │ Yes │
├───────────────────────────────┼───────────────┼───────────────────────────────────────┤
│ External audio libs needed │ No │ Yes │
├───────────────────────────────┼───────────────┼───────────────────────────────────────┤
│ Use case │ Voice calls │ Custom processing, TTS, file playback │
└───────────────────────────────┴───────────────┴───────────────────────────────────────┘
Both modes can run simultaneously (e.g., mic + background music).

---

Test Procedure

1. List audio devices

cd examples
python basic_room.py --list-devices
Expected: Lists available microphones and speakers with device IDs.

2. Connect with PlatformAudio

Start LiveKit server

livekit-server --dev

In another terminal

export LIVEKIT_URL=ws://localhost:7880
export LIVEKIT_API_KEY=devkey
export LIVEKIT_API_SECRET=secret

python basic_room.py --platform-audio --room test-room
Expected: Connects to room, publishes microphone track with voice processing.

3. Test with specific device

python basic_room.py --platform-audio --mic-id "" --room test-room
Expected: Uses specified microphone.

4. Test WAV file playback (synthetic mode)

python basic_room.py --file test.wav --room test-room
Expected: Publishes audio from WAV file.

5. Test mixed mode (PlatformAudio + file)

python basic_room.py --platform-audio --file test.wav --room test-room
Expected: Publishes two audio tracks - microphone and file.

6. Verify with second participant

Open https://meet.livekit.io and join the same room to verify audio is received.

[devin-ai-integration]

Devin Review found 1 potential issue.

View 3 additional findings in Devin Review.

[devin-ai-integration]

🔴 24-bit WAV conversion crashes due to missing sign extension

The 24-bit to 16-bit audio conversion in load_wav_file zero-pads the most-significant byte (+ b"\x00") instead of sign-extending it. For any negative 24-bit sample (where frames[i+2] & 0x80 is set), this produces a large positive 32-bit value. After the >> 8 shift, the result exceeds the signed 16-bit range (-32768..32767), causing struct.pack('h', ...) to raise struct.error. Since roughly half of all samples in typical audio are negative, this will crash immediately on almost any real 24-bit WAV file.

Suggested change

	sample = struct.unpack("<i", frames[i : i + 3] + b"\x00")[0] >> 8
	sample = struct.unpack("<i", frames[i : i + 3] + (b"\xff" if frames[i + 2] & 0x80 else b"\x00"))[0] >> 8

---

Was this helpful? React with 👍 or 👎 to provide feedback.