bstein/lesavka
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
lesavka/docs/opus-transport-testing.md

1.6 KiB
Raw Blame History

Opus Transport Testing

Lesavka does not use SIP, RTP, or SIPp for upstream microphone media. Upstream audio is bundled with webcam video in UpstreamMediaBundle, and the launcher can select either compressed Opus or raw S16LE PCM.

The Opus profile protects the production migration seam:

  • The protobuf schema can explicitly label AudioPacket payloads as PCM_S16LE or OPUS.
  • Legacy or unstamped audio still normalizes to PCM_S16LE.
  • Client microphone packets stamp PCM metadata before optional Opus encoding.
  • Bundled media carries batch-level audio encoding metadata beside video.
  • The server decodes Opus back to raw PCM before handing audio to the UAC sink.
  • If Opus encode/decode is unavailable, Lesavka falls back to or drops safely instead of treating compressed bytes as PCM.
  • Performance tests compare the 20 ms Opus budget against raw PCM byte pressure.

Current Opus model:

  • sample rate: 48000
  • channels: 2
  • frame duration: 20000us
  • target bitrate: 64000bps
  • expected payload: about 160 bytes per 20 ms frame

The current route is:

raw mic capture -> optional webrtcdsp -> opusenc -> gRPC bundle -> opusdec -> raw UAC

PCM remains available as:

raw mic capture -> optional webrtcdsp -> gRPC bundle -> raw UAC

Any recalibration should record both the upstream video codec (HEVC/MJPEG) and upstream audio codec (Opus/PCM) so old PCM baselines stay distinguishable. Server calibration profiles use camera+audio names such as hevc+pcm and hevc+opus. The PCM maps are the known-good factory values; the Opus maps start as a separate namespace and should be replaced only after Theia/RCT calibration.