Worker (hermes-agent) was hard-pinned to titan-08 (a workaround after an earlier
attempt to place it on the amd64 titan-22 failed on architecture). That single-
node pin is exactly what makes it fragile: a titan-08 blip (as just happened when
the node's Longhorn CSI went down) strands worker, and the Recreate strategy then
deadlocks because the replacement can't schedule on the one tight node.
Restore the intended multi-node design: run on any arm64 storage worker except the
known-bad/weak ones (matching the repo's own affinity test, which was red). Its
Longhorn volumes have data-locality disabled with replicas on titan-15/17/19, so
there is no locality penalty to running on another node; the scheduler now places
it on a roomier Pi (e.g. titan-05) and a node blip simply reschedules it.
Also relax the gateway /api/status liveness probe (timeout 10s->15s,
failureThreshold 3->5) so a transient slowness (e.g. a brief storage hiccup) no
longer trips a kill-and-restart cascade.
Follow-up (separate): multi-arch agent image to enable the amd64 titan-22 target.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
The overlay status caption already carries a static ellipsis (e.g. 'Thinking…'),
and after ~2.5s of silent thinking the 'working' affordance added an animated
dots ::after on top of it — so the label intermittently rendered as 'Thinking……'
only on longer turns. Drop the animated text dots and keep the orb-halo shimmer
as the liveness cue, so the ellipsis stays a single, consistent static one.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Conversation mode was only reachable via a small icon by the composer, which is
easy to miss on a fresh session. Inject a prominent, centred 'Start
conversation' button into the empty new-chat state (#emptyState), below the
subtitle and above the suggestions, so it sits in the vertical centre of the
screen. It is created only when local voice is available, honours the same
show/hide preference as the composer toggle, and enters conversation mode
through the same activate() path. Fully guarded so it degrades to nothing if the
empty state is absent. New probe scenario verifies it is created, visible, and
opens the conversation overlay on press.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
On an Android standalone PWA the composer's bottom control row was rendering
about one line below the visible viewport, behind the system gesture bar, so
those controls were unreachable. The theme already pads the titlebar with
env(safe-area-inset-top/left/right), but the viewport meta never opted into
viewport-fit=cover, so every safe-area inset collapsed to 0 and the bottom edge
had no reservation.
Add viewport-fit=cover to the viewport meta (base patch) so the insets carry
real values, and reserve env(safe-area-inset-bottom) at the bottom of the
composer (brand.css), additive with the app's existing --keyboard-bottom-inset
and absorbed by the flex-1 message scroller so total height stays within the
viewport. Inert on desktop (env() resolves to 0). Dockerfile verifies the meta
patch landed; brand/dockerfile contracts covered by tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Diagnosed from a live voice session (e1b9fccb90ef): three turns failed with
raw '**Error:** HTTP 502 ... hermes-{claude,codex}-broker' because the agent
pod hosting the model brokers rolled mid-conversation. The voice client
correctly refused to speak the error envelope, but it then dropped the user's
utterance and forced them to repeat it three times.
Voice: on a TRANSIENT provider error (5xx/502/'error sending request'/timeout),
conversation mode now re-runs the errored turn in place through the app's own
regenerate action (which truncates the errored turn — no duplicate user
message) and stays in Thinking so its cues cover the reconnect gap. Bounded to
MAX_TRANSIENT_RETRIES (2); a non-transient error or an exhausted budget still
drops cleanly to 'let's try that again — listening'. The raw error is never
spoken. New probe scenarios cover retry-then-recover and the bounded-then-drop
path; source contract updated.
STT: the same session mis-transcribed 'CUI' as 'cue'. Prime the default
initial_prompt with the domain acronyms the user uses (CUI, FOUO, DoD, NIST,
CMMC, FIPS, RMF, POA&M, ATO, SBU) so they bias to uppercase forms.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
The orb watermark was showing the character art as a low-opacity square
box (visible edges) — the earlier screen-blend pass washed it into a glow
that lost the artwork. Reprocess the source into a feathered circle so the
orb crops it (no box) while keeping the character's facial features, paint
it with a normal blend at 0.5 opacity, and size it to fill the orb (inset
7%). Verified by compositing over a simulated orb before shipping.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
- REGRESSION FIX: the full-screen conversation overlay stopped opening
(fell back to the inline bar) because building the language/output
selectors inside the overlay's single try/catch could throw on a
phone (navigator.mediaDevices/setSinkId). Overlay build is now two
phases: the essential orb+captions+controls attach first; the
selectors attach after, each guarded, so a selector failure omits only
that control and never the visualization. Control builders can no
longer throw (inert hidden fallback); device enumeration is async
after attach. Regression test covers mediaDevices-undefined and
enumerate-rejects.
- Orb watermark is the processed Hermes character glyph (feathered,
circle-cropped, screen-blended so the face glows on the dark orb), no
more white/grey box.
- Output selector defaults to the loudspeaker (excludes the
communications/earpiece endpoint that the open mic otherwise forces);
explicit choice sticks; hidden gracefully where setSinkId is
unsupported. 291 voice tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Beam-2 with a temperature-fallback ladder made the final-model warmup
run all three temperature retries under beam search before the server
bound its port, so /health was refused for ~8 min and STT was down that
whole time on every roll (and hinted at slow per-utterance decodes).
Production now runs the accurate large-v3-turbo model greedily at a
single temperature, keeping the proper-noun priming prompt that fixes
names like Amy/Córdoba - fast startup, fast decodes, accuracy intact.
Beam stays env-tunable (HERMES_STT_FINAL_BEAM_SIZE) for a future pass;
serve-before-warmup is a recommended follow-up so cold start never
blocks readiness.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
A transcription that landed while the private Whisper service was
restarting (an image roll) crashed hermes_stt_client.py with a raw
urllib ConnectionRefused traceback that got dumped into the
conversation. The client now retries the request with backoff (up to 5
attempts, ~10s - long enough to ride an STT pod restart) and, on a
persistent outage, exits with one concise line ('speech transcription
unavailable...') instead of a stack trace. Delivered via the coordinator
ConfigMap; the next reconcile picks it up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Final conversation-mode polish from mobile testing:
- The Workspace toggle now sits with the chat/Telegram nav at every
width: nav.rail on desktop, the top app titlebar on mobile. The
floating pill that pushed the mobile composer's control row (and the
conversation-mode button) off screen is gone - a fallback exists only
for headless DOMs and is pinned to a top corner, never over the
composer.
- The conversation orb watermark is now the Hermes character avatar
(static/hermes-agent-192.png) instead of the caduceus staff.
- User-facing 'hands-free' copy renamed to 'Conversation mode'.
- Wrong-voice fix: strongReplyLanguage flagged Spanish on a single
accented char, so an English reply naming European cities (Zürich,
Málaga) overrode the correct English STT detection and was spoken by
the Spanish voice. Detection now requires density (Cyrillic >=4 at
>=50%, or inverted punctuation / >=2 accents corroborated by Spanish
stopwords); plain English always speaks English, forced language wins,
accent-free Spanish still routes via trusted STT. 294 voice tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Three conversation-mode fixes in one pass:
- Interim-acknowledgement truncation: when an interim ack folds into the
hidden worklog segment mid-speech, the retained unspoken tail is now
flushed and spoken in full before the Thinking transition, and a
distinct follow-up message is chunked from its own start and queued
after the interim drains (no more 'stops after the first clause, rest
resurfaces with the next message').
- Natural thinking fillers: brief per-language interjections (Umm/Hmm/
One sec; Mmm/A ver; Хм/Секунду) on genuine >1.9s thinking gaps only,
non-repeating, answer-preempting, mute-aware.
- One unified audio sink for every spoken output (reply, cues, fillers,
WAV fallback) - fixes cues playing the loudspeaker while the reply
used a different output - plus a tidy corner output-device selector
(enumerateDevices + setSinkId, feature-detected, session-only) styled
like the language selector. Also realigns two STT-server decode-param
assertions to the dict form from the STT tuning commit. 284 voice tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Default final beam 5 -> 2: the accuracy/speed knee - most of beam
search's benefit at ~2x greedy instead of ~5x, protecting commit
latency on the Jetson (still env-overridable via
HERMES_STT_FINAL_BEAM_SIZE). Prompt now also primes common names (Amy,
Claude, Hermes) so 'Amy' stops transcribing as 'aiming'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Proper nouns (Córdoba, Cancún) and dropped words came from decoding the
committed transcript with the small model. The image already ships
large-v3-turbo, so the final decode now uses it with beam_size=5, a
temperature fallback ladder, and a proper-noun/accents initial_prompt
that fixes first-pass capitalization and diacritics across EN/ES/RU;
the rolling previews stay on tiny at greedy so the on-the-fly feel is
unchanged. The accurate decode runs in the speculative predecode during
the end-of-speech silence and is cache-reused at commit, so perceived
latency stays low. All decode knobs are env-overridable for on-device
tuning (beam/temperature/prompt), with small as the guaranteed-present
rollback if turbo underperforms on the Jetson. 206 STT tests pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
The size cap was applied to the raw wire body, so a nodes list (huge
because of status.images) truncated before the image-stripping ran and
came back as a truncation notice. Accept up to 6 MiB on the wire to
parse and clean, then enforce the 384 KiB model-facing cap on the
stripped result - nodes now returns real data. Delivered via the
cluster-read ConfigMap; picked up on the next pod roll.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
The 'Something went wrong - listening' state with no spoken answer was a
false positive: readAssistantTurn flagged the whole turn as an error if
ANY segment was error-stamped - including a recovered/transient tool
error or a cancellation notice from an earlier interim - and threw away
the real answer that the same turn produced. Error now surfaces only
when the turn yielded no spoken answer at all; a turn with real content
is spoken normally. Softened the genuine-error label to the friendlier
'Let's try that again - listening'. New probe scenarios lock both: an
error segment alongside an answer speaks the answer (error=false), and
an error-only turn still reports the error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Real root cause (confirmed against the live build-24 DOM): the caption
and TTS extraction fell back to turn.textContent whenever a settle-frame
race left no readable answer segment, scraping the avatar letter,
author name and 'Processed 13s' chip - and that truncated reply made
TTS speak only the first segment then drop to Listening even with the
mic muted (the muted-mic first-sentence-stop). Extraction now prefers
each answer segment's data-raw-text, else the answer .msg-body only
(excluding thinking/tool/worklog/role chrome), and the textContent
fallback is gone; a genuinely mid-flight reply retries briefly so the
whole thing is read before the overlay drains. Also: the app's own
caduceus mark embedded in the conversation orb as a subtle watermark; a
corner language selector (Auto + en/es/ru) that forces both the STT
hint and the reply voice; and a thinking affordance after 2.5s of dead
time. New response probe + extraction test prove caption==body-only and
that every sentence reaches the TTS queue. 278 voice-lane tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
The chat prompt still forbade all cluster access, so Hermes told users
it had no visibility even though cluster_read is live - it now knows it
has a read-only cluster tool (no Secrets/Vault) and should use it rather
than deny. Adds tool/research-conditioned acknowledgement guidance: when
a turn needs a lookup, plan or calculation, open with one short 'on it,
~ETA' line then deliver the full answer; simple questions get no
preamble. In spoken mode that first line is read aloud. Fixes two
pre-existing exact-match test pins the HUX/cluster rollout had grown
(plugins.enabled list; a fieldRef env comprehension).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Node .status.images (every cached image on the node) overflowed the
size cap and left json.loads parsing a truncated blob, so a nodes query
came back as a non-JSON error. The de-noise pass now summarizes that
list, and genuine truncation returns the readable prefix with a
narrow-your-query hint instead of an error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
The read-only cluster_read tool (and the HUX-12 producer) reach the
Kubernetes API through the kubernetes Service, which kube-proxy DNATs
from the 10.43.0.1 ClusterIP to a control-plane node on 192.168.22.11-13
:6443 - addresses the tenant egress except-block was dropping, so calls
failed with connection-refused. Egress now allows the ClusterIP and
those three apiserver endpoints on 443/6443. RBAC still bounds what is
readable (no Secrets); nothing else about the isolation changes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
- Captions read message bodies only (the scraper was concatenating
avatar, author and worklog chips); multi-segment interim turns are
now speakable and drive clean speak-to-thinking-to-speak cycles when
playback drains mid-turn.
- Dynamic endpointing: complete-looking partials (3+ words or terminal
punctuation) endpoint at the base window; the long hold remains only
for one-two-word fragments. A stale-busy 10s settle wait on every
post-error send is gone.
- Both overlay captions are bounded, touch-scrollable regions with
follow-tail; caps raised for long turns.
- Error envelopes are never spoken or captioned; errored turns run
resyncCapture (fresh STT session on the hot mic).
- Workspace toggle now lives in the sidebar rail (floating button only
below the rail breakpoint).
- False 'session unavailable' toast root-caused: the router continuity
guard shows it on a 409 that fired when a transient profile-listing
failure failed closed into a fake cross-profile mismatch; the patcher
now answers from the alias cache and never claims a default-vs-named
mismatch while aliases are unconfirmed.
- Barge-in sends carry a one-line cut-point marker with the last spoken
sentence; visible-history truncation judged infeasible client-side.
- Language switching works end-to-end: sticky per-session STT language
hint (restarting an unused next session on switch), reply voice from
script evidence, STT detection, then stopword heuristic; cues and WAV
fallback share the turn language.
- Legacy CI guards: node skip for the DOM probe, ffmpeg/codec skips for
the container-fallback test. 272 voice-lane tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
worker.bstein.dev (the hermes-agent Deployment) gains the same HUX
shape as chat, staged and inert: a foundation-only hux sidecar on the
reviewed WebUI image line (Flux setters bound, 5s probe budgets), an
init that provisions the HMAC identity as slot-100 on the durable home
subtree (create-once context key, O_EXCL subject binding, per-pod
worker key; no relay/router/evidence keys so those trusts fail closed),
and observe-only hook env in the agent container with the runtime
plugin mounted but deliberately NOT enabled - activation is a reviewed
one-line flip per docs/hux/WORKER-PLAN.md, which carries the rollout,
verification gates, canary/rollback ladder and open questions.
Cross-surface continuity remains unclaimed until the live gates pass.
7 new topology-adaptive delivery gates green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Re-enforcement prerequisites, code-side complete: the autonomy runtime
now docks pending-approval cards above the composer (newest first, cap
three, aria-live, allow-once / always / deny wired to the existing
decide route with idempotency; polling gated to active turns and
fail-tolerant), so parked tool calls are never silent. The default
capability matrix no longer denies by default: network and web_search
ask below autonomous (visible prompt) and nothing resolves to deny
except explicit grants or private mode; SO-39 stays intact - deploy and
external side effects always ask and external never auto-allows.
rules.py at 100% line+branch; 744 hux-lane tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
RBAC: the built-in view ClusterRole (which never includes Secrets, so
Vault-managed material stays structurally invisible) plus a read-only
extra for nodes, namespaces, PVs, storage classes, CRDs, Flux resources
and metrics, bound to the chat service account. Tooling: a cluster-read
plugin registers a GET-only cluster_read tool against the in-cluster
API using the pod's projected token - secrets paths refused in the
handler as well, malformed segments rejected, responses bounded and
stripped of managedFields noise. Classified read_files/low in the HUX
capability map. RBAC applies on push; the tool activates when the pods
next roll (bundled with the round-3 voice build).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
The env setters moved to block style so Flux can rewrite them; the
renderer belt now matches the same shape and the release test asserts
the bound value lines rather than the old flow mapping.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Round 2 from live testing:
- Clipping root cause: the endpointer committed on any 1.1s pause, so a
thinking pause after a sentence opener sent one word; a speculative
Whisper pass over that fragment then stalled the real commit ~3.5s on
the Jetson. Young utterances now hold a 1.8s endpoint until 1.2s of
speech accrues, speculative decode waits for 700ms of speech, resume
is unconditional after any 650ms gap (server-frozen snapshots can
never reach commit), and the noise floor is capped so playback echo
cannot deafen onset. Deterministic capture-continuity probe added.
- Stitching now fires for any barge-cancelled send within 20s,
regardless of partial assistant output.
- The workspace-drawer dark rectangle was our own HUX chrome resolving
undefined theme tokens (--bg-primary) to a flat box; bootstrap.css
bridges the real theme tokens and drops a compositor-hazard
backdrop-filter.
- Conversation mode: full-viewport hands-free overlay with an energy-
driven orb (mic RMS in, TTS activity out), state caption, live
transcript/reply captions, mute and exit controls, focus trap,
Escape, reduced-motion support. Voice lane 253 tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
