bstein/lesavka
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
lesavka/AGENTS.md

43 KiB
Raw Blame History

Lesavka Agent Notes

A/V Sync Probe And Lip-Sync Validation Checklist

Context: Google Meet testing on 2026-04-30 showed audio roughly 8 seconds behind video even though internal client/server telemetry reported fresh uplink packets. Treat this as a product correctness failure, not a calibration issue. Do not resume blind lip-sync tuning until the probe can explain where delay appears.

Operating Principles

  • Avoid hard-resetting USB, UVC, UAC, display managers, or remote hosts unless the user explicitly approves it.
  • Prefer observation and reversible user-space probes before changing media pipelines.
  • Treat Tethys-only SSH/device inspection as a development luxury, not a production dependency.
  • Do not claim lip sync is fixed from internal telemetry alone; require end-to-end device-level evidence.
  • Keep this checklist updated as work lands.

Phase 1: Build The Probe

  • Create this tracked checklist in AGENTS.md.
  • Inventory existing client/src/sync_probe/ code and decide what can be reused.
    • Reuse the existing synthetic beacon in client/src/sync_probe/.
    • Reuse the existing Tethys capture harness in scripts/manual/run_upstream_av_sync.sh.
    • Reuse and extend lesavka-sync-analyze; current gap is structured evidence output, not capture generation.
  • Define the phase-1 output contract:
    • report.json
    • report.txt
    • per-event rows with event id, video time, audio time, skew, and confidence
    • pass/fail verdict using preferred/acceptable/catastrophic thresholds
  • Add a deterministic local sync beacon source:
    • video flash pattern with event identity or cadence
    • simultaneous audio click/beep
    • stable event schedule suitable for automated detection
  • Add a Tethys-side capture probe:
    • capture Lesavka UVC video device
    • capture Lesavka UAC microphone device
    • record enough raw evidence for debugging when detection fails
    • detect video flashes
    • detect audio clicks
    • pair events and compute skew
  • Add a runner that can launch or instruct the Tethys probe safely over SSH without rebooting or restarting the desktop.
  • Store probe artifacts under /tmp/lesavka-sync-probe-* by default.
  • Keep the probe usable without Google Meet first; Google Meet validation is a later application-level check.

Phase 2: Use Probe To Root-Cause Desync

  • Run probe through direct Lesavka UVC/UAC devices on Tethys.
    • First live run reached the devices but exposed analyzer/tooling gaps instead of a valid skew report.
    • Fixed the manual probe tunnel to preserve HTTPS/mTLS through SSH (LESAVKA_SERVER_SCHEME=https, LESAVKA_TLS_DOMAIN=lesavka-server).
    • Fixed analyzer handling for MJPEG captures whose FFprobe metadata over-reports frames versus decodable video frames.
  • Compare client-generated event times against Tethys-observed times.
    • The preserved Tethys capture had 323 decodable frames with constant brightness, so no video flash reached UVC.
    • Server logs show the probe entered a stale upstream session and dropped audio as ~326 seconds late.
  • Identify whether delay appears before server planning, at server UAC sink, at UVC helper, inside Tethys device capture, or inside browser/WebRTC.
    • Current root cause is server planning/session lifecycle, before UVC/UAC sink output.
    • A previous one-sided microphone session started at 2026-04-30T22:59:52Z; the new probe at 2026-05-01T00:57:08Z inherited its stale playout epoch.
  • Add diagnostics for whichever stage is hiding delay.
    • Existing server lifecycle/planning logs were enough to isolate this run; next gate should preserve these as structured artifacts.
  • Do not tune calibration offsets until gross backlog is ruled out.
    • No calibration offsets were changed during the stale-session investigation.
    • Current evidence points at lifecycle/session planning, not an offset problem.

Phase 3: Fix Lesavka With Evidence

  • If stale upstream lifecycle is confirmed, reset shared A/V timing anchors when a new stream replaces an existing owner.
    • Added a lifecycle guard so normal camera/microphone stream replacement clears stale shared timing anchors before re-pairing.
    • Kept soft microphone recovery intentionally separate so it supersedes the mic owner without disturbing an active healthy camera/shared clock.
    • Added regression coverage for stale timing-anchor replacement and soft microphone recovery preservation.
  • If UAC sink backlog is confirmed, make UAC output freshness-bounded.
  • If audio progress is marked too early, move/augment progress telemetry to reflect actual sink emission readiness.
  • If UVC and UAC are using incompatible freshness semantics, unify them behind one live-media policy.
  • If browser/WebRTC adds delay after devices are already synced, document the application boundary and add browser-specific mitigation or guidance.

Phase 4: Gate And Release Criteria

  • Add deterministic unit/integration tests for probe analysis logic.
  • Add a hardware-in-the-loop/manual gate artifact schema for real Tethys probe runs.
  • Update scripts/ci/media_reliability_gate.sh to report probe evidence when present.
    • Gate now reads LESAVKA_SYNC_PROBE_REPORT_JSON, LESAVKA_SYNC_PROBE_REPORT_DIR, or target/media-reliability-gate/sync-probe/report.json.
    • Gate emits sync-probe verdict/check metrics, skew metrics, event counts, and a verdict info metric.
  • Require a fresh probe report before declaring lip sync fixed.
    • Gate now supports LESAVKA_REQUIRE_SYNC_PROBE=1, which fails media reliability when a valid passing probe report is absent.
    • Product/release judgment still requires a new live Theia/Tethys probe after the lifecycle fix is installed.
  • Suggested thresholds:
    • preferred: p95 skew <= 35 ms
    • acceptable: p95 skew <= 80 ms
    • gross failure: sustained skew > 250 ms
    • catastrophic failure: any sustained skew near or above 1000 ms

Open Questions

  • Decide whether the phase-1 beacon should run as a separate binary, a hidden client mode, or both.
  • Decide whether Tethys probe should be Rust-only, shell plus GStreamer, or a hybrid.
  • Confirm whether sudo/Vault access is available for installing missing probe dependencies on Theia/Tethys.
    • Non-sudo server journal inspection worked; noninteractive sudo over SSH still needs an explicit TTY/password path.

Validation Evidence

  • cargo test -p lesavka_server upstream_media_runtime::tests::lifecycle
  • cargo test -p lesavka_client sync_probe::analyze
  • cargo test -p lesavka_testing upstream_sync_script_tunnels_auto_server_addr_through_ssh
  • bash -n scripts/ci/media_reliability_gate.sh
  • cargo test -p lesavka_testing media_reliability_gate_reports_direct_sync_probe_evidence
  • LESAVKA_REQUIRE_SYNC_PROBE=1 ./scripts/ci/media_reliability_gate.sh
    • Used a synthetic passing report at target/media-reliability-gate/sync-probe/report.json to verify gate parsing/enforcement.
    • This validates CI glue only; a real Theia/Tethys probe is still required for product judgment.

Real Upstream Lip-Sync Fix Checklist

Context: the mirrored browser probe finally reproduced the real failure class on 2026-05-01: activity_start_delta_ms=+9591.1. This means the end-to-end browser-visible path can still start video far ahead of audio. The fix target is not silence in the logs; it is a freshness-first A/V uplink whose startup can heal briefly but cannot drift into seconds of skew.

Acceptance Criteria

  • Mirrored browser probe passes with activity_start_delta_ms <= 1000.
  • Steady-state preferred sync: median skew within 35 ms.
  • Steady-state acceptable sync: p95 absolute skew within 80 ms.
  • Any sustained or startup A/V split near 1000 ms remains a hard failure.
  • No stale audio backlog is ever drained into UAC to catch up.
  • No stale video backlog is ever drained into UVC to catch up.
  • Google Meet manual testing agrees with the mirrored probe instead of revealing hidden seconds-scale skew.

Phase 0: Keep The Probe Honest

  • Split raw activity-start fields from filtered/coded paired-pulse fields in probe reports.
  • Print explicit raw first-video and first-audio timestamps in report.txt.
  • Root-cause the 0.16.17 raw_first_video_activity_s=0.000 artifact as the mirrored probe counting its own bright pre-start positioning card.
  • Make the mirrored stimulus pre-start screen dark/dim so only real flash pulses can be detected as video activity.
  • Add analyzer coverage proving dim pre-start positioning frames are ignored.
  • Replace generic light/dark mirrored flashes with color-coded event IDs.
  • Make mirrored audio pulses unique by the same event ID via pulse width plus tone frequency.
  • Teach the analyzer to decode mirrored video event IDs from color, not grayscale brightness.
  • Tighten real-camera color matching after 0.16.18 accepted washed-out brown/gray remnants as red/yellow events.
  • Preserve raw activity-start timing before cadence cleanup in coded reports.
  • Merge short audio envelope dropouts inside one coded pulse so a single tone burst cannot become two fake events.
  • Add diagnostic coded-pair correlation so stable large skew reports as measured failure instead of not enough pairs.
  • Make coded mirrored verdicts/calibration use matched coded pulses as authority; raw activity-start deltas are reported separately unless they agree with the coded pairs.
  • Print unpaired video/audio onsets in the human report so missed coded pulses are visible during probe triage.
  • Keep the mirrored browser probe as the release/blocking upstream A/V gate.
  • Keep the old raw-device probe as a lower-level diagnostic only.

Phase 1: Stop One-Sided Startup Drift

  • Default upstream planning must require both camera and microphone before live playout.
  • One-sided playout may only happen through an explicit compatibility override.
  • While pairing is overdue, keep replacing the waiting-side anchor with fresh packets instead of preserving stale startup anchors.
  • While awaiting the peer stream, keep only fresh pending camera packets.
  • While awaiting the peer stream, keep only fresh pending microphone packets.
  • Send the latest camera packet from the client uplink queue instead of draining old-but-not-yet-stale video backlog.
  • Add tests proving the pairing window no longer expires into one-sided playout by default.
  • Add tests proving the explicit one-sided override still works for intentional single-stream scenarios.

Phase 2: Bound UAC Freshness

  • Configure UAC appsrc as non-blocking and bounded.
  • Log and drop UAC appsrc push failures instead of treating enqueue as guaranteed playback.
  • Raise calibration offset limits enough to cover the measured MJPEG/UVC path delta without rejecting probe corrections.
  • Update the MJPEG/UVC factory audio baseline from the old -45ms/+720ms values to +1260ms as the mirrored probe exposes the fresh UAC-vs-UVC path delta.
  • Migrate untouched legacy -45ms factory/env calibration files on load so old installs actually receive the new baseline.
  • Make the video/audio-master wait offset-aware so a positive audio playout delay does not freeze UVC video while UAC sleeps before emission.
  • Flush/stop UAC cleanly on session close, replacement, and recovery.
  • Add tests or contract coverage for bounded UAC settings where practical.

Phase 3: Add Real Timing Evidence

  • Add server timing counters for first camera packet, first mic packet, first UVC write, and first UAC push per session.
  • Add dropped-stale audio/video counters to diagnostics.
  • Add a concise health explanation when startup pairing exceeds the healing window.
  • Surface Starting, Healing, Flowing, Lagging, Dropping, and Stale states in chips/diagnostics from real path evidence.

Phase 4: Recovery And Mid-Session Changes

  • Make device changes trigger soft-pause, stream replacement, queue flush, and re-pairing.
  • Keep recovery soft-first; reserve hard UVC/UAC gadget rebuilds for explicit guarded recoveries.
  • Add cooldown/state guards so recovery buttons cannot wedge Theia.
  • Ensure disconnect closes all client/server media tasks for the session.

Phase 5: Verification Loop

  • Run focused upstream runtime tests.
  • Run server/client media contract tests.
  • Run cargo check for touched packages.
  • Bump version for the fix release.
  • Run the mirrored browser probe on installed client/server.
    • 0.16.17 still failed: reported activity_start_delta_ms=+6735.0, but raw_first_video_activity_s=0.000 exposed a probe false-positive from the pre-start screen. Paired pulses still showed real steady-state skew (p95=411.8 ms, median=-99.0 ms), so the product remains unfixed.
    • 0.16.18 captured real colored/audio-coded events but the analyzer still bailed with need at least 3 matching coded pulse pairs; saw 1. Replaying that artifact after analyzer hardening now reports gross_failure: 16/16 coded pairs, p95 775.7 ms, activity start -766.4 ms, and drift -2.8 ms; the failure is stable audio-ahead/video-late skew, not random detector noise.
    • 0.16.19 changes the shipped MJPEG/UVC audio playout baseline to +720ms; the next mirrored browser probe should move the measured median from about -766ms toward roughly -46ms before fine calibration.
    • 0.16.19 mirrored browser probe did not move the measured skew: p95 885.7 ms, median -788.4 ms, activity start -659.1 ms, drift -81.2 ms. SSH inspection showed Theia was on commit c348597, but /etc/lesavka/server.env still contained LESAVKA_UPSTREAM_AUDIO_PLAYOUT_OFFSET_US=-45000; the new +720ms baseline was not actually installed. Patch the installer to migrate leaked legacy ambient -45000 to +720000 unless LESAVKA_INSTALL_UPSTREAM_AUDIO_PLAYOUT_OFFSET_US explicitly asks for the legacy value.
    • 0.16.20 installed the +720ms offset (/etc/lesavka/server.env had LESAVKA_UPSTREAM_AUDIO_PLAYOUT_OFFSET_US=720000), but the mirrored browser capture contained no recognizable color pulses. Theia server logs showed repeated upstream video frame dropped because the audio master never caught up inside the pairing window; UVC was effectively starved by the positive audio delay instead of flowing delayed-but-fresh frames.
    • 0.16.21 makes that wait offset-aware and adds a regression test proving a configured positive audio delay does not freeze UVC video while UAC sleeps before playout.
    • Replaying the 0.16.21 artifact after 0.16.22 analyzer hardening changes the verdict from false catastrophic_failure to gross_failure: p95 273.8 ms, median -188.4 ms, 7 paired coded pulses. The raw activity-start delta (-3620.7 ms) is still printed, but it is ignored for verdict/calibration because it disagrees with coded pairs by 3432.3 ms; unpaired video/audio onsets are printed for triage.
    • 0.16.22 live mirrored run still failed with p95 433.7 ms, median -359.4 ms, and 5 paired coded pulses. Client telemetry showed camera uplink latest_age_ms repeatedly around 300-350 ms, matching the measured skew; patch 0.16.23 to make video queues latest-only instead of draining stale-but-under-budget backlog.
    • 0.16.23 local validation passed for fresh-queue behavior, uplink/probe freshness contracts, sync analyzer tests, client/server binary checks, and whitespace checks.
    • 0.16.23 live mirrored run improved to p95 215.2 ms, median +142.2 ms, 13 paired coded pulses, and raw activity alignment within 6.6 ms of coded pairs. Patch 0.16.24 makes the probe print local client and remote server versions before capture so every run records what was actually tested.
    • 0.16.24 live mirrored run improved again to p95 168.4 ms, median -19.1 ms, 11 paired coded pulses, but still failed because individual paired pulses bounced between about -168 ms and +45 ms. Client logs showed the microphone uplink queue still accumulating depth 16; patch 0.16.25 makes microphone uplink queues latest-only too so stale audio PTS cannot continue acting as the server timing master under backpressure.
    • 0.16.25 removed the client mic backlog but exposed a stable hardware/browser path delta: p95 557.3 ms, median -540.5 ms, drift +9.0 ms, and fresh mic delivery ages around 2-10 ms. Patch 0.16.26 raises the MJPEG/UVC factory audio delay to +1260 ms and expands the calibration clamp so this stable offset can actually be corrected instead of rejected.
  • Re-run the mirrored browser probe after the pre-start false-positive fix.
  • Run Google Meet manual validation.

0.17.0 Tyrannical Upstream Playout Checklist

Context: 0.16.x proved that queue tweaks and static calibration cannot guarantee lip sync. 0.17.0 changes the upstream contract: the server planner is authoritative, audio is the master, video follows by timestamp or freezes, and freshness wins over smooth-but-wrong playback.

Hard Product Invariants

  • No normal live upstream playout may be more than 1000 ms behind the freshest known client capture frontier.

  • Video may not advance outside the audio playhead sync budget.

  • Audio should be continuous when possible, but stale audio must be dropped/skipped rather than drained.

  • Missing or late video should freeze/stutter instead of pulling audio backward.

  • Startup may be ugly, but must either enter fresh synced live mode or declare failure within 60 seconds.

  • Healing may be visible, but must prevent persistent seconds-scale skew.

  • Calibration may fine-tune sub-frame offsets only; it must not be required to rescue seconds-scale desync.

  • Bump Lesavka to 0.17.0 because this is a media-contract change, not a patch tune.

  • Bump patch follow-ups to 0.17.1 instead of reporting version+revision as the release version.

  • Add planner policy config: max live lag, max skew, startup timeout, target playout delay, and healing cooldown.

  • Reset 0.17 defaults so shipped audio/video offsets do not intentionally exceed the freshness budget.

  • Track latest camera/audio input timestamps in the server planner.

  • Track actual planned/emitted audio and video playheads.

  • Enforce audio as master: stale audio is dropped/skipped; it does not drain backlog.

  • Enforce video follower behavior: frames ahead of audio wait; frames too far behind audio are dropped so the previous frame freezes.

  • Re-anchor continuously when the live playhead falls outside the freshness budget, not only once at startup.

  • Keep startup paired-only by default and fail visibly after the startup timeout.

  • Add planner phases and counters to diagnostics/logs: acquiring, syncing, live, healing, failed; stale drops, skew drops, freshness heals, video freezes.

  • Keep UVC/UAC sinks as dumb consumers of planner-approved packets.

  • Update tests to prove stale media cannot be emitted and video cannot outrun/lag audio beyond policy.

  • Update manual/probe diagnostics so 0.17 reports the planner state being tested.

Validation Targets Before Human Test

  • Unit tests for startup pairing, stale audio drop, stale video drop, reanchor, startup timeout, audio-master/video-follower rules.
  • Contract tests for installer defaults and version reporting.
  • cargo check -p lesavka_client -p lesavka_server --bins.
  • Focused lesavka_testing media/runtime contracts.
  • Only after all of the above, run the mirrored browser probe.

Progress Log

  • 2026-05-01: Added 0.17 planner defaults (350ms target playout, 1000ms max live lag, 60000ms startup timeout, 80000us pair slack), reset MJPEG audio factory offset to 0, and migrated old -45ms, +720ms, and +1260ms untouched baselines.
  • 2026-05-01: Server planner now tracks latest input frontier, presented audio/video playheads, phase, stale drops, skew drops, reanchors, startup timeouts, and freezes.
  • 2026-05-01: Runtime tests green for stale audio drop, stale video drop, audio-master/video-follower freeze, repeated reanchor, paired startup timeout, and planner snapshot basics: cargo test -p lesavka_server upstream_media_runtime::tests -- --nocapture.
  • 2026-05-01: Added GetUpstreamSync RPC, lesavka-relayctl upstream-sync, launcher diagnostics text, and mirrored-probe before/after planner snapshots so 0.17 probe runs report the exact planner state under test.
  • 2026-05-01: Validation green: cargo test -p lesavka_server --lib --bins, cargo test -p lesavka_testing, cargo test -p lesavka_client --bins --lib, and targeted installer/RPC/layout contracts.
  • 2026-05-01: First installed 0.17.0 mirrored browser probe on client/server commit 3920e0a failed honestly: planner reported fresh live state (live_lag_ms=10, skew_ms=+20.7) but browser-observed paired pulses showed audio late by median +349.1ms, p95 429.1ms, with 6 video freezes/skew drops. Replayed artifact after analyzer hardening now reports gross_failure instead of false raw-start catastrophic_failure.
  • 2026-05-01: Patch follow-up models the observed MJPEG/UVC browser egress delta by defaulting video playout offset to +350ms and preserving the 1s freshness ceiling. Raw activity-start evidence is now ignored for verdict/calibration when it disagrees with paired pulses that are already failing directly. Existing early-0.17 audio=0/video=0 factory/env calibration files migrate to the new video=+350ms default on load.
  • 2026-05-01: Release identity cleanup: bumped the patched build to clean semver 0.17.1; probe attribution now prints client_version/server_version separately from client_revision/server_revision and refuses old client_full_version output.
  • 2026-05-01: 0.17.1 mirrored probe failed with video about 1.18-1.31s behind audio and 761 planner video freezes. Root cause candidate: the client rebaser forced independent camera/mic pipelines onto one first-packet capture base, so a later-starting camera path was timestamped too early and looked permanently behind audio. Patch 0.17.2 anchors each stream to the shared monotonic clock at its own first packet time.
  • 2026-05-02: 0.17.2 mirrored probe and Google Meet test showed major improvement but persistent sub-second late video. Root cause follow-up: the temporary +350ms factory MJPEG video playout offset matched the observed browser skew and also made the server skew guard freeze video against its own offset. Patch 0.17.3 restores factory video offset to 0ms, migrates untouched +350ms install/calibration defaults back to 0ms, and makes the skew guard offset-aware for intentional site calibration.
  • 2026-05-02: 0.17.3 Google Meet manual test improved to roughly sub-second/near-quarter-second lip sync, but the mirrored analyzer could not pair pulses and the user still heard choppy background audio. Client logs showed Pulse microphone packets arriving unevenly with ages around 90-240ms; patch 0.17.4 lowers Pulse mic buffer-time/latency-time, bounds the mic queue/appsink, and keeps mirrored-probe after-run planner diagnostics even when analysis fails.
  • 2026-05-02: 0.17.4 mirrored run was salvageable after an SCP banner timeout, but analysis still failed with no close pulse pairs. The client log still showed 180-240ms microphone delivery ages, pointing at server playout sleeps backpressuring the gRPC microphone stream. Patch 0.17.5 drains inbound microphone packets while waiting for scheduled UAC playout and retries browser-capture SCP fetches.
  • 2026-05-02: 0.17.5 mirrored run still failed with insufficient paired evidence, and the client log still showed recurring 180-240ms microphone packet age while camera age stayed near zero. Patch 0.17.6 splits oversized mic samples into 20ms timestamped packets and keeps a short fresh server-side audio window instead of collapsing every pending burst to one newest chunk, aiming to preserve lip sync without making background audio choppy.
  • 2026-05-02: 0.17.6 Bumblebee mirrored run proved Bumblebee mic packets are already 10ms, but camera source timestamps were being rebased up to roughly 1.8s into the future while mic packets sat around 180-240ms old. Patch 0.17.7 adds a source lead cap (80ms default) to both direct and duration-paced client timestamp rebasing so bursty camera buffers cannot make the server wait for fake future video while fresh audio keeps moving.
  • 2026-05-02: The launcher UI was still writing live control files with only camera/mic/speaker booleans, so media device combo changes were honestly only staged for the next child launch. Patch 0.17.7 extends the live media control file with base64-encoded camera source, camera profile, microphone source, and speaker sink choices; the relay child now rebuilds the affected camera, mic, or speaker pipeline when those selections change.

0.17.8 Sync-Only Output Bias Checklist

Context: 0.17.7 with the Bumblebee mic and BRIO camera removed the seconds-scale failure and left a stable browser-visible output skew: paired pulses were audio-late by roughly +95ms to +183ms (median=+110.8ms, mean=+132.6ms, p95=+183.1ms). Per user direction, 0.17.8 is only about establishing sync. Freshness and smoothness tuning are explicitly deferred until the mirrored probe is inside the sync band.

  • Do not change freshness ceilings, reanchor thresholds, queue policy, UAC smoothness, or startup healing behavior in this version.
  • Set the MJPEG/UVC factory video playout baseline to +130ms to counter the measured browser output audio-late bias.
  • Migrate only untouched old 0ms and +350ms video defaults to the new +130ms baseline.
  • Preserve manual/site calibration values exactly as-is.
  • Update installer defaults so Theia receives the same +130ms baseline after reinstall.
  • Update docs and contracts to state the measured sync baseline clearly.
  • Run focused calibration/installer/runtime tests.
  • Run package checks before push.
  • Push clean semver 0.17.8 for installed client/server testing.

0.17.9 Sync-Only Audio-Master Presentation Checklist

Context: 0.17.8 installed cleanly on both ends (314c55b) but the mirrored probe failed with insufficient data: only 2 paired events, 1187 video freezes, and planner phase healing. The server was using the newest planned audio packet as the video-drop reference, so future audio planning could make current video look falsely behind before that audio was actually handed to UAC.

  • Keep 0.17.9 scoped to sync enforcement only; no freshness ceilings, queue policy, or smoothness changes.
  • Make video freeze/drop decisions compare against audio actually presented to UAC, not merely planned audio.
  • Make wait_for_audio_master wake on mark_audio_presented so video waits for real audio progress.
  • Add/adjust tests proving future planned audio alone cannot freeze video.
  • Run focused upstream planner tests.
  • Run package checks before push.
  • Push clean semver 0.17.9 for installed client/server testing.

0.17.10 Sync-Only Audio Catch-Up Grace Checklist

Context: 0.17.9 installed cleanly on both ends (fbf274d) and improved the mirrored probe to median=+19.8ms, mean=-42.0ms, and planner phase live, but it still failed with p95=254.1ms, only 6 paired pulses, drift=341.9ms, and 591 video freezes. The Theia server log showed repeated upstream video frame dropped because the audio master never caught up inside the pairing window, so the video follower was still giving up at the nominal video due time instead of spending a bounded sync grace to let audio catch up.

  • Keep 0.17.10 scoped to establishing sync; defer freshness and smoothness tuning until paired skew is stable.
  • Add LESAVKA_UPSTREAM_AUDIO_MASTER_WAIT_GRACE_MS with a 350ms default so video can wait past nominal due time for UAC audio progress.
  • Stop dropping video solely because it woke late after a successful audio-master wait.
  • Preserve the global 1000ms live-lag ceiling and existing stale-input planner rules.
  • Update installer defaults and operational docs for the sync grace.
  • Add/adjust tests proving video can wait through sync grace and still times out after grace expires.
  • Run focused upstream planner tests.
  • Run package checks before push.
  • Push clean semver 0.17.10 for installed client/server testing.

0.17.11 Sync-Only Browser Egress Compensation Checklist

Context: 0.17.10 installed cleanly on both ends (4bb0f4a) and produced a high-confidence coded-pulse failure instead of probe ambiguity. Browser-visible audio on Tethys arrived about +891ms to +971ms after the matching video (median=+962.1ms, mean=+946.7ms, p95=+971.5ms), while the server planner reported internal skew near zero (planner_skew_ms=-56.9). The missing model is UAC/browser output egress latency: Lesavka was treating appsrc.push_buffer/UAC enqueue as audio presentation, but the browser consumes that audio about one second later.

  • Keep 0.17.11 scoped to establishing sync; do not tune freshness ceilings or smoothness policy.
  • Raise the MJPEG/UVC factory video playout baseline from +130ms to +1090ms to align video with browser-visible UAC audio.
  • Allow intentional A/V playout offsets to exceed the generic future-wait freshness guard so the planner does not immediately reanchor away the sync compensation.
  • Widen calibration offset bounds so the measured browser egress baseline is representable instead of silently clamped.
  • Migrate untouched 0ms, +130ms, and +350ms MJPEG/UVC video baselines to the new browser-visible baseline.
  • Preserve manual/site calibration values exactly as-is.
  • Update installer defaults so Theia receives the same browser-visible baseline after reinstall.
  • Update operational docs and installer contract tests for the new baseline.
  • Run focused calibration, installer, and runtime checks.
  • Push clean semver 0.17.11 for installed client/server testing.

0.17.12 Sync-Only Audio Marker Continuity Checklist

Context: 0.17.11 installed cleanly on both ends (092c03a) and fixed the constant browser-visible offset: raw activity start was -17.0ms, median coded skew was +13.1ms, and no server planner freeze/drop counters moved. The run still failed because only 5 coded pairs were usable and the last two had weak confidence (0.58, 0.33) while the client log showed repeated microphone stale/superseded drops. The remaining sync blocker is audio-marker continuity, not another constant offset.

  • Keep 0.17.12 scoped to sync establishment; do not change video freshness or server freshness ceilings.
  • Stop using latest-only policy for microphone uplink packets; preserve oldest fresh audio chunks so speech/pulses stay continuous.
  • Keep microphone queue age bounded at 400ms so continuity does not become unbounded backlog.
  • Apply the same bounded audio-continuity policy to the synthetic sync-probe audio queue.
  • Update contract tests to encode the new audio continuity policy.
  • Run focused client queue/probe contracts and package checks.
  • Push clean semver 0.17.12 for installed client/server testing.

0.17.13 Probe-Driven Calibration Loop Checklist

Context: 0.17.12 installed cleanly on both ends (2b26fde) and moved the paired-pulse median into the near-sync zone (median=+71.6ms, first/last +59.8ms/-60.1ms), but the run still failed with only 4 usable coded pairs and p95=206.8ms. Static guesses have reached diminishing returns. 0.17.13 adds a measured calibration loop so the mirrored probe can become the authority for site-specific browser-visible output compensation when, and only when, the analyzer has enough stable evidence.

  • Keep 0.17.13 scoped to probe-driven sync calibration tooling; do not change freshness ceilings, queue policy, UAC smoothness, or startup healing behavior.
  • Expose relay CLI calibration state and safe calibration actions for scripts.
  • Make the mirrored probe locate the analyzer report.json after a run and print the calibration decision.
  • Add opt-in probe calibration apply mode gated by LESAVKA_SYNC_APPLY_CALIBRATION=1.
  • Apply only when analyzer calibration.ready=true; otherwise refuse and print the analyzer reason.
  • Default probe-driven correction to video offset adjustment, because the measured residual is browser-visible UAC egress delay relative to UVC video.
  • Keep saving the measured offset as the site default opt-in via LESAVKA_SYNC_SAVE_CALIBRATION=1.
  • Update contract tests for relay CLI and manual probe script behavior.
  • Run focused relay/manual-script tests and package checks.
  • Push clean semver 0.17.13 for installed client/server testing.

Follow-up candidate: after 0.17.13 proves safe measured apply/refuse behavior, add a segmented live-calibration probe. The current browser probe uploads one WebM after recording ends, so it can only do measure/apply/rerun. A true same-session loop should run a longer stimulus, capture/analyze separate Tethys browser windows, apply calibration only between windows, and use the next window as the confirmation segment so before/after evidence is not mixed.

0.17.14 Segmented Live Calibration Probe Checklist

Context: 0.17.13 adds safe measured calibration apply/refuse plumbing, but it is still a single-window measure-then-rerun workflow. The next probe should keep the same Lesavka client/server session alive across multiple browser-capture windows so we can measure, apply, and re-measure without reinstalling or restarting the media path. This is the bridge from probe truth to blind/server-side calibration targets.

  • Keep 0.17.14 scoped to probe tooling and observability; do not change media planner policy.
  • Add optional multi-segment mirrored probe mode via LESAVKA_SYNC_CALIBRATION_SEGMENTS.
  • Keep one local stimulus browser and one headless Lesavka sender alive across all segments.
  • Run a fresh Tethys browser recording/analyzer pass per segment so before/after calibration evidence is not mixed in one WebM.
  • Allow calibration apply between segments using the 0.17.13 ready/refuse gate.
  • Capture planner and calibration snapshots before and after each segment for metric correlation.
  • Preserve single-segment default behavior for normal manual probes.
  • Update manual probe contract tests for segmented live calibration mode.
  • Run focused script/CLI checks and package checks.
  • Push clean semver 0.17.14 for installed client/server testing.

0.17.15 Adaptive Probe Metrics and Blind Target Checklist

Context: 0.17.14 can keep one Lesavka session alive across multiple measured segments, but we still need the probe to teach Lesavka what "good" looks like from server-only telemetry. 0.17.15 turns segmented runs into an adaptive calibration dataset: every segment gets probe truth, planner state, and calibration state joined into artifacts that can drive blind calibration/healing targets when Tethys/browser probe access is not available.

  • Keep 0.17.15 scoped to probe intelligence and metrics correlation; do not change media playout policy.
  • Add adaptive calibration ergonomics for longer near-continuous runs without changing the default one-segment probe.
  • Write per-run segment metrics as CSV and JSONL, joining analyzer verdicts with planner/calibration before/after snapshots.
  • Emit a blind-target candidate JSON from segments whose probe verdict passes, including server-visible planner lag/skew ranges.
  • Record when no segment is probe-good enough so blind-target generation refuses instead of inventing targets.
  • Keep calibration mutation gated by the existing ready/refuse logic and LESAVKA_SYNC_APPLY_CALIBRATION=1.
  • Update manual probe contract tests for the adaptive artifacts and controls.
  • Run focused script checks and package checks.
  • Push clean semver 0.17.15 for installed client/server testing.

0.17.16 Continuous Browser Evidence Checklist

Context: 0.17.15 proved the adaptive/live-edit loop is structurally useful, but each segment restarted the Tethys browser/getUserMedia receiver. That contaminates calibration segments with receiver startup noise and keeps usable coded pairs too low (3-5 pairs instead of the 8+ needed for safe calibration). 0.17.16 is scoped to making the probe evidence continuous and attributable before any more media playout changes.

  • Keep 0.17.16 scoped to probe/tooling reliability; do not change server media playout policy, freshness ceilings, queue policy, or UAC smoothness.
  • Make adaptive mirrored runs keep one Tethys browser/getUserMedia session alive after the first segment.
  • Preserve single-segment/manual probe behavior by default.
  • Add an explicit BROWSER_CONSUMER_REUSE_SESSION control to the browser probe runner.
  • Verify browser uploads by start token so the fetched WebM belongs to the segment just triggered.
  • Track browser upload counts/tokens in browser_consumer_probe.py status JSON for post-run debugging.
  • Wire adaptive mirrored mode to reuse the existing Tethys receiver for segments 2+.
  • Keep calibration mutation behind the existing analyzer calibration.ready=true gate.
  • Update manual probe contract tests for continuous browser session behavior.
  • Run shell syntax checks, focused contract tests, and package checks.
  • Push clean semver 0.17.16 for installed client/server testing.

0.17.17 Adaptive Segment Failure Continuation Checklist

Context: 0.17.16 successfully installed and token-verified the first browser capture, but the adaptive run still aborted at segment 1 because lesavka-sync-analyze could not form coded pairs (saw 0) even though raw activity was nearly synced (-32.2ms). That prevented segments 2-4 from exercising the same long-lived Tethys browser session. 0.17.17 keeps adaptive data gathering alive across analyzer-only failures and preserves the failure evidence for summaries.

  • Keep 0.17.17 scoped to probe/tooling reliability; do not change media playout policy.
  • Add BROWSER_ANALYSIS_REQUIRED so browser captures can keep artifacts even when analyzer exits nonzero.
  • In adaptive mirrored mode, treat analyzer-only failures as nonfatal segment evidence.
  • Preserve analyzer failure reason, raw activity delta, and log path as structured segment artifacts.
  • Continue to abort on browser startup, recording, upload, or capture fetch failures.
  • Include analyzer failure artifacts in segment CSV/JSONL summaries.
  • Keep calibration apply impossible without a real analyzer report.json and calibration.ready=true.
  • Update manual probe contract tests for analyzer-failure continuation.
  • Run shell syntax checks, focused contract tests, and package checks.
  • Push clean semver 0.17.17 for installed client/server testing.

0.17.18 Explicit Probe Source Integrity Checklist

Context: the 0.17.17 Bumblebee mirrored run was configured with LESAVKA_MIC_SOURCE=alsa_input.usb-Neat_Microphones_Bumblebee_II_USB_Microphone-00.mono-fallback, but the Bumblebee was unplugged. The client log recorded requested mic ... not found; using default, so the run measured the fallback/default microphone and must not be treated as Bumblebee calibration evidence.

  • Treat 0.17.17 Bumblebee probe metrics as invalid for Bumblebee-specific calibration.
  • Keep ordinary launcher/client source selection forgiving by default.
  • Add strict explicit-source mode with LESAVKA_REQUIRE_EXPLICIT_MEDIA_SOURCES=1.
  • In strict mode, fail client startup when requested LESAVKA_MIC_SOURCE is unavailable instead of falling back to default.
  • Make the mirrored manual probe launch the real client with strict explicit-source mode by default.
  • Add contract coverage so the mirrored probe cannot regress to silent explicit-source fallback.
  • Run shell syntax checks, focused contract tests, and package checks.
  • Push clean semver 0.17.18 for installed client/server testing.
  • Re-run the mirrored probe only after confirming the intended microphone is physically present and selected.

0.17.19 Fatal Required Source Failure Checklist

Context: the 0.17.18 run proved fallback was blocked, but the headless client kept running camera-only after the required Bumblebee source failed to open. The server stayed in acquiring for all four segments (awaiting both upstream media streams), the analyzer saw no color-coded video pulses, and no calibration data was produced. Required-source failure must fail the probe, not degrade into camera-only evidence.

  • Treat the 0.17.18 run as a required-microphone setup failure, not a lip-sync measurement.
  • Keep strict no-fallback behavior from 0.17.18.
  • Abort the client process when an explicit required microphone source cannot start.
  • Abort the client process when an explicit required camera source cannot start.
  • Run shell syntax checks, focused contract tests, and package checks.
  • Push clean semver 0.17.19 for installed client/server testing.
  • Re-run only after LESAVKA_MIC_SOURCE is listed by the local audio stack.

0.17.20 Provisional Adaptive Calibration Checklist

Context: the 0.17.19 adaptive run reached live media and produced browser-visible sync evidence, but the probe still did not calibrate. Each segment either had only 3 coded pairs or an analyzer failure, so the existing calibration.ready=true gate refused every adjustment. That was safe for saved defaults, but wrong for the user-directed adaptive probe goal: live segments should be allowed to make bounded provisional corrections from imperfect-but-usable evidence, then let the next segment judge whether the correction helped.

  • Treat the 0.17.19 run as a measurement-only run, not a calibration run.
  • Keep permanent saved calibration gated by analyzer-ready evidence.
  • Add provisional adaptive calibration mode enabled by adaptive runs by default.
  • Allow provisional corrections from low-but-usable paired-pulse reports when p95 and drift are bounded.
  • Use median skew as the provisional correction source and default to video offset adjustment.
  • Clamp provisional correction gain and max step so one noisy segment cannot swing the site offset wildly.
  • Record decision mode and provisional recommendations in segment metrics.
  • Update manual probe contract tests for provisional calibration controls and output.
  • Run shell syntax checks, focused contract tests, and package checks.
  • Push clean semver 0.17.20 for installed client/server testing.

0.17.21 Calibrate-Then-Confirm Probe Checklist

Context: 0.17.20 made adaptive runs capable of provisional calibration between measured segments, but that still did not strictly guarantee the user-requested flow: run the probe, calibrate the server while it is running, then run a post-calibration test segment. It also still ignored analyzer-failure captures that contained a bounded raw activity delta. 0.17.21 makes the probe behavior explicit: calibration segments mutate active server calibration, confirmation segments do not mutate it, and adaptive runs fail unless confirmation passes.

  • Treat LESAVKA_SYNC_CALIBRATION_SEGMENTS as calibration windows in adaptive confirm mode.
  • Add post-calibration confirmation windows via LESAVKA_SYNC_CONFIRMATION_SEGMENTS.
  • Disable calibration apply during confirmation windows so they are a clean test.
  • Require confirmation pass by default in adaptive confirm mode.
  • Add bounded raw-activity provisional calibration for analyzer failures that still report a raw A/V delta.
  • Include confirmation summaries and segment phase in adaptive artifacts.
  • Update manual probe contract tests for calibrate-then-confirm behavior.
  • Run shell syntax checks, focused contract tests, and package checks.
  • Push clean semver 0.17.21 for installed client/server testing.