40 lines
1.6 KiB
Markdown
40 lines
1.6 KiB
Markdown
|
# Opus Transport Testing
|
||
|
|
|
||
|
|
Lesavka does not use SIP, RTP, or SIPp for upstream microphone media. Upstream
|
||
|
|
audio is bundled with webcam video in `UpstreamMediaBundle`, and the launcher can
|
||
|
|
select either compressed `Opus` or raw `S16LE` PCM.
|
||
|
|
|
||
|
|
The Opus profile protects the production migration seam:
|
||
|
|
|
||
|
|
- The protobuf schema can explicitly label `AudioPacket` payloads as `PCM_S16LE`
|
||
|
|
or `OPUS`.
|
||
|
|
- Legacy or unstamped audio still normalizes to `PCM_S16LE`.
|
||
|
|
- Client microphone packets stamp PCM metadata before optional Opus encoding.
|
||
|
|
- Bundled media carries batch-level audio encoding metadata beside video.
|
||
|
|
- The server decodes Opus back to raw PCM before handing audio to the UAC sink.
|
||
|
|
- If Opus encode/decode is unavailable, Lesavka falls back to or drops safely
|
||
|
|
instead of treating compressed bytes as PCM.
|
||
|
|
- Performance tests compare the 20 ms Opus budget against raw PCM byte pressure.
|
||
|
|
|
||
|
|
Current Opus model:
|
||
|
|
|
||
|
|
- sample rate: `48000`
|
||
|
|
- channels: `2`
|
||
|
|
- frame duration: `20000us`
|
||
|
|
- target bitrate: `64000bps`
|
||
|
|
- expected payload: about `160 bytes` per 20 ms frame
|
||
|
|
|
||
|
|
The current route is:
|
||
|
|
|
||
|
|
`raw mic capture -> optional webrtcdsp -> opusenc -> gRPC bundle -> opusdec -> raw UAC`
|
||
|
|
|
||
|
|
PCM remains available as:
|
||
|
|
|
||
|
|
`raw mic capture -> optional webrtcdsp -> gRPC bundle -> raw UAC`
|
||
|
|
|
||
|
|
Any recalibration should record both the upstream video codec (`HEVC`/`MJPEG`)
|
||
|
|
and upstream audio codec (`Opus`/`PCM`) so old PCM baselines stay distinguishable.
|
||
|
|
Server calibration profiles use `camera+audio` names such as `hevc+pcm` and
|
||
|
|
`hevc+opus`. The PCM maps are the known-good factory values; the Opus maps start
|
||
|
|
as a separate namespace and should be replaced only after Theia/RCT calibration.
|