titan/atlas-iac
3
0
Fork 0
Code Issues Pull Requests 9 Actions Packages Projects Releases Wiki Activity

WIP: Voice mode: speak replies in the language private Whisper STT detected (follows #26) #27

Closed
hermes-automation wants to merge 1 commits from hermes/voice-stt-tts-language-routing into main
pull from: hermes/voice-stt-tts-language-routing
merge into: titan:main
Conversation 0 Commits 1 Files Changed 9 +1359 -15
hermes-automation commented 2026-08-20 20:11:38 +00:00
Member
Copy Link

Follows PR #26 — merge this after it

This is a separate, independent branch cut fresh from origin/main at
11bd04cce5d8861bb9d4865c580d77ffac48e218. It does not contain, amend,
rebase onto, or merge any commit from PR #26 (hermes/tts-voice-multilingual),
and I have not touched, closed or merged that PR.

PR #26 bakes three Piper voices and adds the server-side language routing
that picks between them (English to amy, Russian to irina, Spanish to claude,
everything else to amy). It states plainly that the WebUI never sends a
language, so in practice every reply is spoken by the English voice. This
PR supplies that missing signal.

Merge order matters. Land #26 first, then this. If this lands alone the
change is inert but harmless: today's hermes-tts server reads only
input/text and speed from the request body, so the extra language field
is ignored and everything still speaks English.

Note on a stale branch ref. An earlier ref
hermes/voice-stt-language-routing exists at fe45d23e from a first push. The
SCM broker only permits ref creation — it rejects updates and deletions — so I
could not move it and could not remove it. That commit is missing the one-line
test_hermes_chat_quality.py fix described under "Files changed", so it is
superseded and should be ignored. This PR's head branch is the one to review;
no PR was ever opened against the stale ref.

Expected merge conflict, and how to resolve it. Both PRs edit the same four
lines of the Atlas TTS request body in dockerfiles/hermes-webui-atlas-patch.py.
#26 deletes the dead hardcoded "voice" line; this PR adds a "language" line
next to it and deliberately leaves "voice" alone so its diff stays purely
additive against main. The resolution is to keep both changes — drop the
"voice" entry, keep "language": _atlas_tts_language(data). A botched
resolution cannot ship silently: the patch is a fail-closed replace_exact and
Dockerfile.hermes-webui greps for the exact resulting text, so the image build
fails instead. A smaller conflict may also appear in services/hermes/NOTES.md
if #26's new voice section lands adjacent to this one; both sections should be
kept.

What I actually traced first

Whether this needs a WebUI patch at all depends entirely on what
/api/transcribe and the local-command STT contract actually look like, so I
traced them before designing anything. It does need one.

I pulled the two pinned upstream images by digest and read the real code rather
than guessing:

  • WebUI ghcr.io/nesquena/hermes-webui at digest
    sha256:a83a3893111dcb250e7aa7aa657d3d6f4570b0e2fd00d9b7569246fc5e7339b2
    (the digest Dockerfile.hermes-webui pins).
  • Agent nousresearch/hermes-agent at digest
    sha256:47d4bd4cc420b70e40ed75efdade373e45b86b7382d4013a054208982bb6ba08
    (the digest Dockerfile.hermes-agent pins). Its
    tools/transcription_tools.py is byte-identical to the copy in this worker
    (sha256:d7d7df56b98dfadc0a7c08d5addd02db4e3106e6fd3b753e9241c23145eb2951),
    and Dockerfile.hermes-agent does not patch that file.

What the path actually is:

  1. atlas-voice.js POSTs the audio blob to /api/transcribe.
  2. That is handle_transcribe in api/upload.py, not api/routes.py. It calls
    tools.transcription_tools.transcribe_audio(temp_path) and returns
    exactly {'ok': True, 'transcript': transcript} — the language is
    dropped even if the provider knew it.
  3. With stt.provider: local_command, transcribe_audio dispatches to
    _transcribe_local_command, whose contract with the shelled-out command is
    "leave a .txt in --output-dir". It globs *.txt, reads the first
    match, and returns {success, transcript, provider}. There is no field in
    that contract that can carry a language    , which is why this could not be
    done by changing only hermes_stt_client.py.
  4. hermes_stt_client.py posts to the Jetson Whisper service, which runs
    auto-detection (stt.local.language: auto in chat-configmap.yaml) but
    returned only {text, model}.
  5. sendTranscript puts the text in the composer and calls the app's own
    window.send(). The reply is spoken later by speakResponse, reached
    through the window.autoReadLastAssistant override — so the language has to
    survive from the transcription response to a callback that happens after a
    full model turn, which is where all the staleness risk lives.

So: a narrow WebUI patch is required, and one narrow agent-side patch with it.
Both are confined to the WebUI image.

The signal path, and what enforces it at each hop

Every hop narrows the value; no hop widens it.

Hop File What it accepts
Whisper result hermes-jetson-stt-server.py result["language"] only as a bare ISO-639 token (2–3 ASCII letters, case/whitespace normalised); anything else reported as empty
Command output hermes_stt_client.py re-validates, writes a <stem>.language sidecar; never a second .txt
Agent envelope patched _transcribe_local_command globs *.language, re-validates the shape, adds language to its result dict
/api/transcribe patched handle_transcribe re-validates, returns language beside transcript
Browser atlas-voice.js keeps it only if it is one of en/ru/es, bound to the voice-mode generation token; consumed once
/api/tts patched Atlas branch fixed allow-list ("en", "ru", "es"), everything else becomes English
Jetson TTS PR #26 the fixed allow-list again, as the final authority

The TTS language is derived only from the server response representing the
STT model result. It is never taken from a voice field — the browser does not
send one and _atlas_tts_language() only ever reads body["language"]. The
browser-side filter is a UX narrowing, not the security control; the server-side
allow-list in api/routes.py is, and PR #26's Jetson-side allow-list is the
final one.

Staleness, races and multi-turn

The language is bound to the voice-mode generation token, which already
increments on every activate/deactivate, and it rides alongside the existing
thinkingSession check. It is read through takeSttLanguage(token), which
returns it only for a matching token and clears it in the same call, so it can
be consumed at most once. It is also cleared explicitly on: deactivation, a new
activation, the start of every listening window, an empty transcript, a session
change between asking and answering, and a transcription error (which
deactivates). The failure direction is always "fall back to English", never
"reuse the previous turn's language".

Why the agent file is patched, and how narrowly

/opt/hermes/tools/transcription_tools.py is patched from
Dockerfile.hermes-webui, not Dockerfile.hermes-agent. That confines it to
the WebUI image. HERMES_LOCAL_STT_COMMAND is set on the webui container only
(chat-statefulset.yaml:353), so that container is both the only consumer of
this path and the only one built from the patched image. The hermes container
is untouched. The edit is additive — one extra key in the returned dict and a
glob for a suffix nothing else writes — so an unpatched caller behaves exactly
as before.

What this does not claim

  • It is the language the user spoke, not the language of the reply. A model
    asked a Russian question may answer in English and will then be read by the
    Russian voice, and the reverse. That is the deliberate policy for hands-free
    mode, not a detection failure.
  • Nothing here detects the language of assistant text, and nothing guesses a
    language from a transcript's characters.
  • Typed messages, the manual read-aloud button, and any assistant reply not
    produced by a spoken turn have no trusted STT signal and stay on the English
    voice. The read-aloud path in ui.js is unchanged and still sends no
    language; there is a test asserting that.
  • /api/transcribe reports what Whisper detected, including languages with no
    baked voice (French comes back as fr). The browser and both server-side
    boundaries then resolve it to English. I chose truthful reporting at the STT
    endpoint and policy enforcement at the TTS boundary rather than blanking the
    field early, so the endpoint does not lie about what the model decided.
  • All of this is documented in services/hermes/NOTES.md under "Private voice:
    choosing the Piper voice from the STT-detected language".

Tests

Written before the implementation; the first run was red on 41 of them.

New testing/tests/test_hermes_voice_language_routing.py — 90 tests, all
passing, no GPU, no microphone, no cluster:

  • Whisper response carries the detected language; shape validation against
    case, whitespace, region tags, path traversal, NUL bytes, wrong types,
    non-ASCII and over-long values.
  • The command client writes the sidecar and provably does not add a second
    .txt (which would otherwise be read as the transcript, since Hermes takes
    the first glob match).
  • The real image patch is applied to fixture trees carrying the exact
    upstream anchors, and the patched modules are then imported and executed —
    so these assert the behaviour of the code that actually ships, not a
    re-implementation of it. Covers the agent envelope, the /api/transcribe
    response, and the Atlas TTS branch.
  • Allow-list enforcement at /api/tts, including a battery of hostile inputs;
    _atlas_tts_language() is proven to return a member of the three-language set
    for every one of them, and voice is proven unable to override language.
  • The browser contract is exercised by running the real atlas-voice.js
    inside a stub DOM, fake MediaRecorder/AudioContext and a fake clock
    (testing/tests/data/atlas_voice_language_probe.js), driving complete turns
    and recording what actually reaches /api/tts: English, Russian and Spanish
    turns each speak in the right language; missing and unsupported languages omit
    the field; hostile values are dropped; three consecutive turns
    (Russian, none, Spanish) do not leak into each other; an empty transcript arms
    nothing; a session change and a deactivation both discard it; a transcription
    error speaks nothing at all.

Independent verification I ran outside the suite, since fixtures cannot prove
anchor fidelity against upstream:

  • Applied both hermes-webui-atlas-patch.py and hermes-webui-router-patch.py
    to the real files extracted from the two pinned image digests. Both
    applied clean, each anchor occurs exactly once, all five new Dockerfile greps
    match, and py_compile passes on the patched routes.py, upload.py,
    gateway_chat.py and transcription_tools.py.
  • AST-checked the patched real routes.py: the helper lands at module scope and
    the branch that calls it is inside the function that binds data.

Other checks: kustomize build services/hermes renders clean and contains the
updated client script; ruff check passes on every changed Python file;
node --check passes on both JS files.

Full suite, measured against a baseline I ran myself. I did not assume which
failures were pre-existing — I ran pytest testing/ on a throwaway detached
worktree of clean origin/main@11bd04cc and compared:

  • baseline origin/main: 2537 passed, 4 failed, 4 skipped
  • this branch: 2627 passed, 4 failed, 4 skipped

Same four failures, same tests, in both:
test_hermes_agent_layout.py::test_cli_lane_reserves_cpu_headroom_for_ui_and_auth,
test_hermes_agent_layout.py::test_agent_avoids_unhealthy_nodes_and_fits_its_remaining_capacity,
test_hermes_gitea_pr_integration.py::test_http_error_path_redacts_token,
test_hermes_kanban_supervisor.py::test_auto_supervise_flag_defaults_false_in_configmap.
None touch voice, STT or TTS code. The 4 skips are cluster-access and visudo
checks this sandbox cannot run. The first run of this branch had a fifth failure
that the baseline did not; that was a real regression I introduced, and fixing
it is the one-line test_hermes_chat_quality.py change described below.

Rollout order

No deployment and no image digest bump in this PR.

Three images are involved. Dockerfile.hermes-jetson-tts belongs to #26;
Dockerfile.hermes-jetson-stt and Dockerfile.hermes-webui belong to this PR.
hermes_stt_client.py is not in any image — it ships through the
hermes-coordinator ConfigMap with disableNameSuffixHash: true, mounted at
/opt/coordinator and re-executed per transcription, so it takes effect on the
kubelet's next volume sync without a pod restart.

  1. Merge PR #26 first.
  2. Merge this PR, resolving the request-body conflict as described above.
  3. Rebuild all three images from the merged main.
  4. Bump the hermes-tts and hermes-stt digests in
    services/hermes/voice-deployment.yaml and the hermes-webui digest in
    services/hermes/chat-statefulset.yaml. None of these have Flux image
    automation, so each is a manual bump.
  5. Let Flux reconcile, then confirm a spoken Russian turn comes back in the
    Russian voice, a spoken English turn in English, and a typed message in
    English.

The three components in this PR can roll in any order and in any partial
combination: a missing language at any hop degrades to English rather than
failing, so a half-rolled state is never broken, only not-yet-multilingual. The
only genuinely order-sensitive step is #26's own manifest-versus-image hazard,
which that PR documents.

Files changed

  • dockerfiles/hermes-jetson-stt-server.py
  • dockerfiles/hermes-webui-atlas-patch.py
  • dockerfiles/hermes-webui-atlas-voice.js
  • dockerfiles/Dockerfile.hermes-webui
  • services/hermes/scripts/hermes_stt_client.py
  • services/hermes/NOTES.md
  • testing/tests/test_hermes_voice_language_routing.py (new)
  • testing/tests/data/atlas_voice_language_probe.js (new)
  • testing/tests/test_hermes_chat_quality.py — one line

That last one is not optional and I want to flag it, because PR #26 edits the
same file. test_chat_voice_uses_private_jetson_services_and_shared_auto_route
pins the exact text pending=fetchSpeech(chunks[index+1]) to assert the chunk
prefetch pipelining. fetchSpeech now takes the language as a second argument,
so that assertion is updated to
pending=fetchSpeech(chunks[index+1],language) — same intent, new signature.
I caught it because the first full-suite run on this branch had exactly one
failure that the origin/main baseline did not. #26's edits to this file are
checksum/env/annotation assertions in different tests, so this should merge
cleanly, but it is a second file both PRs touch.

## Follows PR #26 — merge this **after** it This is a separate, independent branch cut fresh from `origin/main` at `11bd04cce5d8861bb9d4865c580d77ffac48e218`. It does **not** contain, amend, rebase onto, or merge any commit from PR #26 (`hermes/tts-voice-multilingual`), and I have not touched, closed or merged that PR. PR #26 bakes three Piper voices and adds the server-side `language` routing that picks between them (English to amy, Russian to irina, Spanish to claude, everything else to amy). It states plainly that the WebUI never sends a `language`, so in practice every reply is spoken by the English voice. **This PR supplies that missing signal.** **Merge order matters.** Land #26 first, then this. If this lands alone the change is inert but harmless: today's `hermes-tts` server reads only `input`/`text` and `speed` from the request body, so the extra `language` field is ignored and everything still speaks English. **Note on a stale branch ref.** An earlier ref `hermes/voice-stt-language-routing` exists at `fe45d23e` from a first push. The SCM broker only permits ref *creation* — it rejects updates and deletions — so I could not move it and could not remove it. That commit is missing the one-line `test_hermes_chat_quality.py` fix described under "Files changed", so it is superseded and should be ignored. This PR's head branch is the one to review; no PR was ever opened against the stale ref. **Expected merge conflict, and how to resolve it.** Both PRs edit the same four lines of the Atlas TTS request body in `dockerfiles/hermes-webui-atlas-patch.py`. #26 deletes the dead hardcoded `"voice"` line; this PR adds a `"language"` line next to it and deliberately leaves `"voice"` alone so its diff stays purely additive against `main`. The resolution is to keep both changes — drop the `"voice"` entry, keep `"language": _atlas_tts_language(data)`. A botched resolution cannot ship silently: the patch is a fail-closed `replace_exact` and `Dockerfile.hermes-webui` greps for the exact resulting text, so the image build fails instead. A smaller conflict may also appear in `services/hermes/NOTES.md` if #26's new voice section lands adjacent to this one; both sections should be kept. ## What I actually traced first Whether this needs a WebUI patch at all depends entirely on what `/api/transcribe` and the local-command STT contract actually look like, so I traced them before designing anything. It does need one. I pulled the two pinned upstream images by digest and read the real code rather than guessing: - WebUI `ghcr.io/nesquena/hermes-webui` at digest `sha256:a83a3893111dcb250e7aa7aa657d3d6f4570b0e2fd00d9b7569246fc5e7339b2` (the digest `Dockerfile.hermes-webui` pins). - Agent `nousresearch/hermes-agent` at digest `sha256:47d4bd4cc420b70e40ed75efdade373e45b86b7382d4013a054208982bb6ba08` (the digest `Dockerfile.hermes-agent` pins). Its `tools/transcription_tools.py` is byte-identical to the copy in this worker (`sha256:d7d7df56b98dfadc0a7c08d5addd02db4e3106e6fd3b753e9241c23145eb2951`), and `Dockerfile.hermes-agent` does not patch that file. What the path actually is: 1. `atlas-voice.js` POSTs the audio blob to `/api/transcribe`. 2. That is `handle_transcribe` in `api/upload.py`, not `api/routes.py`. It calls `tools.transcription_tools.transcribe_audio(temp_path)` and returns **exactly** `{'ok': True, 'transcript': transcript}` — the language is dropped even if the provider knew it. 3. With `stt.provider: local_command`, `transcribe_audio` dispatches to `_transcribe_local_command`, whose contract with the shelled-out command is *"leave a `.txt` in `--output-dir`"*. It globs `*.txt`, reads the first match, and returns `{success, transcript, provider}`. **There is no field in that contract that can carry a language**, which is why this could not be done by changing only `hermes_stt_client.py`. 4. `hermes_stt_client.py` posts to the Jetson Whisper service, which runs auto-detection (`stt.local.language: auto` in `chat-configmap.yaml`) but returned only `{text, model}`. 5. `sendTranscript` puts the text in the composer and calls the app's own `window.send()`. The reply is spoken later by `speakResponse`, reached through the `window.autoReadLastAssistant` override — so the language has to survive from the transcription response to a callback that happens after a full model turn, which is where all the staleness risk lives. So: a narrow WebUI patch is required, and one narrow agent-side patch with it. Both are confined to the WebUI image. ## The signal path, and what enforces it at each hop Every hop narrows the value; no hop widens it. | Hop | File | What it accepts | |---|---|---| | Whisper result | `hermes-jetson-stt-server.py` | `result["language"]` only as a bare ISO-639 token (2–3 ASCII letters, case/whitespace normalised); anything else reported as empty | | Command output | `hermes_stt_client.py` | re-validates, writes a `<stem>.language` sidecar; **never** a second `.txt` | | Agent envelope | patched `_transcribe_local_command` | globs `*.language`, re-validates the shape, adds `language` to its result dict | | `/api/transcribe` | patched `handle_transcribe` | re-validates, returns `language` beside `transcript` | | Browser | `atlas-voice.js` | keeps it only if it is one of `en`/`ru`/`es`, bound to the voice-mode generation token; consumed once | | `/api/tts` | patched Atlas branch | fixed allow-list `("en", "ru", "es")`, everything else becomes English | | Jetson TTS | PR #26 | the fixed allow-list again, as the final authority | The TTS language is derived **only** from the server response representing the STT model result. It is never taken from a `voice` field — the browser does not send one and `_atlas_tts_language()` only ever reads `body["language"]`. The browser-side filter is a UX narrowing, not the security control; the server-side allow-list in `api/routes.py` is, and PR #26's Jetson-side allow-list is the final one. ### Staleness, races and multi-turn The language is bound to the voice-mode `generation` token, which already increments on every activate/deactivate, and it rides alongside the existing `thinkingSession` check. It is read through `takeSttLanguage(token)`, which returns it only for a matching token and clears it in the same call, so it can be consumed at most once. It is also cleared explicitly on: deactivation, a new activation, the start of every listening window, an empty transcript, a session change between asking and answering, and a transcription error (which deactivates). The failure direction is always "fall back to English", never "reuse the previous turn's language". ## Why the agent file is patched, and how narrowly `/opt/hermes/tools/transcription_tools.py` is patched from `Dockerfile.hermes-webui`, not `Dockerfile.hermes-agent`. That confines it to the WebUI image. `HERMES_LOCAL_STT_COMMAND` is set on the `webui` container only (`chat-statefulset.yaml:353`), so that container is both the only consumer of this path and the only one built from the patched image. The `hermes` container is untouched. The edit is additive — one extra key in the returned dict and a glob for a suffix nothing else writes — so an unpatched caller behaves exactly as before. ## What this does *not* claim - It is the language the **user spoke**, not the language of the reply. A model asked a Russian question may answer in English and will then be read by the Russian voice, and the reverse. That is the deliberate policy for hands-free mode, not a detection failure. - Nothing here detects the language of assistant text, and nothing guesses a language from a transcript's characters. - Typed messages, the manual read-aloud button, and any assistant reply not produced by a spoken turn have no trusted STT signal and stay on the English voice. The read-aloud path in `ui.js` is unchanged and still sends no `language`; there is a test asserting that. - `/api/transcribe` reports what Whisper detected, including languages with no baked voice (French comes back as `fr`). The browser and both server-side boundaries then resolve it to English. I chose truthful reporting at the STT endpoint and policy enforcement at the TTS boundary rather than blanking the field early, so the endpoint does not lie about what the model decided. - All of this is documented in `services/hermes/NOTES.md` under "Private voice: choosing the Piper voice from the STT-detected language". ## Tests Written before the implementation; the first run was red on 41 of them. New `testing/tests/test_hermes_voice_language_routing.py` — **90 tests, all passing, no GPU, no microphone, no cluster:** - Whisper response carries the detected language; shape validation against case, whitespace, region tags, path traversal, NUL bytes, wrong types, non-ASCII and over-long values. - The command client writes the sidecar and provably does **not** add a second `.txt` (which would otherwise be read as the transcript, since Hermes takes the first glob match). - The **real image patch** is applied to fixture trees carrying the exact upstream anchors, and the patched modules are then imported and executed — so these assert the behaviour of the code that actually ships, not a re-implementation of it. Covers the agent envelope, the `/api/transcribe` response, and the Atlas TTS branch. - Allow-list enforcement at `/api/tts`, including a battery of hostile inputs; `_atlas_tts_language()` is proven to return a member of the three-language set for every one of them, and `voice` is proven unable to override `language`. - The browser contract is exercised by **running the real `atlas-voice.js`** inside a stub DOM, fake MediaRecorder/AudioContext and a fake clock (`testing/tests/data/atlas_voice_language_probe.js`), driving complete turns and recording what actually reaches `/api/tts`: English, Russian and Spanish turns each speak in the right language; missing and unsupported languages omit the field; hostile values are dropped; three consecutive turns (Russian, none, Spanish) do not leak into each other; an empty transcript arms nothing; a session change and a deactivation both discard it; a transcription error speaks nothing at all. Independent verification I ran outside the suite, since fixtures cannot prove anchor fidelity against upstream: - Applied both `hermes-webui-atlas-patch.py` and `hermes-webui-router-patch.py` to the **real** files extracted from the two pinned image digests. Both applied clean, each anchor occurs exactly once, all five new Dockerfile greps match, and `py_compile` passes on the patched `routes.py`, `upload.py`, `gateway_chat.py` and `transcription_tools.py`. - AST-checked the patched real `routes.py`: the helper lands at module scope and the branch that calls it is inside the function that binds `data`. Other checks: `kustomize build services/hermes` renders clean and contains the updated client script; `ruff check` passes on every changed Python file; `node --check` passes on both JS files. **Full suite, measured against a baseline I ran myself.** I did not assume which failures were pre-existing — I ran `pytest testing/` on a throwaway detached worktree of clean `origin/main@11bd04cc` and compared: - baseline `origin/main`: **2537 passed, 4 failed, 4 skipped** - this branch: **2627 passed, 4 failed, 4 skipped** Same four failures, same tests, in both: `test_hermes_agent_layout.py::test_cli_lane_reserves_cpu_headroom_for_ui_and_auth`, `test_hermes_agent_layout.py::test_agent_avoids_unhealthy_nodes_and_fits_its_remaining_capacity`, `test_hermes_gitea_pr_integration.py::test_http_error_path_redacts_token`, `test_hermes_kanban_supervisor.py::test_auto_supervise_flag_defaults_false_in_configmap`. None touch voice, STT or TTS code. The 4 skips are cluster-access and `visudo` checks this sandbox cannot run. The first run of this branch had a fifth failure that the baseline did not; that was a real regression I introduced, and fixing it is the one-line `test_hermes_chat_quality.py` change described below. ## Rollout order No deployment and no image digest bump in this PR. Three images are involved. `Dockerfile.hermes-jetson-tts` belongs to #26; `Dockerfile.hermes-jetson-stt` and `Dockerfile.hermes-webui` belong to this PR. `hermes_stt_client.py` is not in any image — it ships through the `hermes-coordinator` ConfigMap with `disableNameSuffixHash: true`, mounted at `/opt/coordinator` and re-executed per transcription, so it takes effect on the kubelet's next volume sync without a pod restart. 1. Merge PR #26 first. 2. Merge this PR, resolving the request-body conflict as described above. 3. Rebuild all three images from the merged `main`. 4. Bump the `hermes-tts` and `hermes-stt` digests in `services/hermes/voice-deployment.yaml` and the `hermes-webui` digest in `services/hermes/chat-statefulset.yaml`. None of these have Flux image automation, so each is a manual bump. 5. Let Flux reconcile, then confirm a spoken Russian turn comes back in the Russian voice, a spoken English turn in English, and a typed message in English. The three components in *this* PR can roll in any order and in any partial combination: a missing `language` at any hop degrades to English rather than failing, so a half-rolled state is never broken, only not-yet-multilingual. The only genuinely order-sensitive step is #26's own manifest-versus-image hazard, which that PR documents. ## Files changed - `dockerfiles/hermes-jetson-stt-server.py` - `dockerfiles/hermes-webui-atlas-patch.py` - `dockerfiles/hermes-webui-atlas-voice.js` - `dockerfiles/Dockerfile.hermes-webui` - `services/hermes/scripts/hermes_stt_client.py` - `services/hermes/NOTES.md` - `testing/tests/test_hermes_voice_language_routing.py` (new) - `testing/tests/data/atlas_voice_language_probe.js` (new) - `testing/tests/test_hermes_chat_quality.py` — one line That last one is not optional and I want to flag it, because PR #26 edits the same file. `test_chat_voice_uses_private_jetson_services_and_shared_auto_route` pins the exact text `pending=fetchSpeech(chunks[index+1])` to assert the chunk prefetch pipelining. `fetchSpeech` now takes the language as a second argument, so that assertion is updated to `pending=fetchSpeech(chunks[index+1],language)` — same intent, new signature. I caught it because the first full-suite run on this branch had exactly one failure that the `origin/main` baseline did not. #26's edits to this file are checksum/env/annotation assertions in different tests, so this should merge cleanly, but it is a second file both PRs touch.
hermes-automation added 1 commit 2026-08-20 20:11:39 +00:00 
Hands-free voice mode had no language signal at all, so every spoken reply was
synthesized with the English voice no matter what the user actually said. The
multilingual Piper work (PR #26) added server-side routing for a "language"
field but nothing ever sent one.

Carry the language the private Jetson Whisper service already detects through
to the TTS request for the reply that speech produced, and only for that reply.

  hermes-stt returns {text, model, language}, accepted only as a bare ISO-639
  token; hermes_stt_client.py writes a <stem>.language sidecar next to the .txt
  transcript Hermes reads, leaving the local-command contract intact; the
  patched local-command envelope and /api/transcribe re-validate it and surface
  it; atlas-voice.js binds it to the voice-mode generation token and chat
  session, consumes it exactly once, and clears it on cancellation, restart,
  session change, empty transcript or transcription error; /api/tts honours it
  only from the fixed en/ru/es allow-list and otherwise sends English.

A client "voice" field is never read at any hop, and typed messages, the manual
read-aloud button, and any reply not produced by a spoken turn carry no trusted
signal and stay on the English voice.

The two WebUI-side and one agent-side edits are fail-closed replace_exact
patches; both patch roots are now env-overridable so the contract can be
verified offline without a GPU or an image build.
bstein closed this pull request 2026-08-22 20:21:34 +00:00

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
atlas-reconciler bstein hermes-automation
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: titan/atlas-iac#27
No description provided.
Delete Branch "hermes/voice-stt-tts-language-routing"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?