384 Commits

Author SHA1 Message Date
jenkins
87c0a8b201 fix(monitoring): bound Titan test metric labels 2026-09-02 23:50:39 -03:00
jenkins
8d1302765f gitea: migrate sources to titan/atlas-iac 2026-09-01 20:43:50 -03:00
jenkins
391a7f2f1f hermes(agent): make titan-22 the strong primary home
Now that both the agent image (a68d1c4d, via the kustomize images: override)
and the hux sidecar (build-39) are multi-arch with amd64 leaves, move the worker
onto the amd64 accelerator titan-22:
- Add an OR'd nodeSelectorTerm for amd64 + node-role.kubernetes.io/accelerator +
  hostname titan-22, with NO worker=true requirement. Keep the arm64 pi-fleet
  term as an OR'd fallback so the worker is never stranded.
- Strong primary preference: hostname=titan-22 at weight 100 (scheduler max),
  pi-fleet rpi5 nudge lowered to 50, so hermes actually lives on titan-22.
- Tolerate node-role.kubernetes.io/accelerator=true:NoSchedule (harmless where
  absent) and the soft atlas.bstein.dev/media-primary:PreferNoSchedule that
  titan-22 currently carries, so the weight-100 preference is not offset and
  placement is deterministic.

Completes the titan-22 effort the flip branches missed; the earlier branches
never repointed to multi-arch images, which is why the worker never landed here.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-25 20:17:28 -03:00
jenkins
975399cbe0 test(monitoring): guard Claude renewal delivery path 2026-08-25 20:06:09 -03:00
jenkins
2c91aea01d fix(hermes-webui): verify OCI revision label on multi-arch index children
The webui release handoff verified org.opencontainers.image.revision on the
Harbor artifact's own extra_attrs.config.Labels. That works for a single-arch
image, but a multi-arch manifest list has no top-level config, so Harbor reports
the label on each per-arch child. build-38 built + published the index fine, then
failed post-publish with 'Harbor artifact omitted OCI image labels'.

verify_registry_digest now checks the top-level config labels when present
(single-arch, unchanged) and otherwise walks the index references, fetching each
child artifact by digest and asserting its revision label. Mirrors how the agent
image lane already tolerates a multi-arch index, without dropping the supply-chain
label check. Adds multi-arch pass/reject tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-25 19:37:27 -03:00
jenkins
db446c9244 monitoring(ai): alert before Claude quota auth expires 2026-08-25 19:16:01 -03:00
jenkins
4908e1ea37 test(hermes): track agent base repoint to in-cluster multi-arch mirror
test_gateway_image_honors_ui_model_and_caps_reasoning still pinned the old
arm64-only docker.io base (nousresearch/hermes-agent@sha256:47d4bd4c...). The
agent image moved to the multi-arch mirror base
(harbor-core.harbor.svc.cluster.local/mirror/hermes-agent@sha256:9c841866...)
in 8a710845 for the two-leg build; the test wasn't updated, so it was a latent
red only the webui/quality lane runs. Point the assertion at the current base.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-25 19:04:58 -03:00
jenkins
342677dde7 monitoring(ai): preserve quota data across rollouts 2026-08-25 18:33:18 -03:00
jenkins
9c516b9808 build(hermes-webui): multi-arch image (arm64 + amd64)
Make registry.bstein.dev/bstein/hermes-webui a linux/amd64 + linux/arm64
manifest list so the agent pod's `hux` sidecar (which runs the webui image)
can schedule onto the amd64 node titan-22. Reuses the hermes-agent multi-arch
pattern already on main.

- Dockerfile.hermes-webui: repoint both FROMs to multi-arch, internal sources.
  The upstream WebUI base (ghcr sha256:a83a3893..., already a multi-arch OCI
  index) is now pulled from the in-cluster Harbor mirror; the agent base moves
  from the retired arm64-only leaf (81970563) to the multi-arch agent index
  (a68d1c4d). Kaniko selects the matching arch leaf per build node.
- services/harbor/hermes-webui-base-mirror-job.yaml: new suspended, operator-run
  skopeo `copy --all` Job mirroring the upstream WebUI base index into Harbor's
  `mirror` project (modeled on hermes-agent-base-mirror-job.yaml; reuses the
  generic ensure-project helper). Wired into the harbor kustomization.
- Jenkinsfile.hermes-webui-image: arm64 leg (titan-20) + amd64 leg (titan-24,
  hostname+arch pin, toleration Exists, resource-capped, own checkout scm) +
  Combine multi-arch index stage; per-arch evidence archived alongside the index.
- hermes_multiarch_combine.py: generalize the destination pattern/component to
  serve both hermes-agent and hermes-webui (fail-closed to just those two).
- Tests updated to the two-arch topology (two legs, combine, both FROM bases,
  the mirror Job, twelve archived evidence files).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-25 18:17:35 -03:00
jenkins
70002aeff7 Revert "Reapply "hermes(agent): make titan-22 the strong primary home (no worker label)""
This reverts commit 9e4fbf4e0df364230588749f80710a8369a3108f.
2026-08-25 17:49:44 -03:00
jenkins
9e4fbf4e0d Reapply "hermes(agent): make titan-22 the strong primary home (no worker label)"
This reverts commit f8628e6ee0d2c33d287ec9086ff5328d982e88c3.
2026-08-25 17:40:36 -03:00
jenkins
12a6d2c4f5 hermes(agent): make runtime tooling install architecture-aware
The hermes-agent installs its CLI toolchain at runtime into the shared
/opt/data/tools Longhorn volume, but every download hardcoded arm64. On
the amd64 node titan-22 that left configure-agent-clients failing with
"Missing optional dependency @openai/codex-linux-x64" and the operator
toolchain fetching arm64 binaries, so the pod churned.

Detect the running node's arch (uname -m; fail closed on anything but
aarch64/x86_64) and resolve every asset per-arch:

- install-agent-tools init script (agent-deployment.yaml): ttyd and
  kubectl download the arch-correct asset with the arch-correct sha256
  (real ttyd 1.7.7 x86_64 and kubectl v1.33.3 amd64 checksums added; the
  arm64 ones kept). The npm CLI stamp is now arch-specific
  (.cli-versions-<vers>-${arch}) so a fresh arch re-runs npm install and
  pulls its own native optional deps; npm keeps both arches' packages.

- install_agent_tools.sh: flux/helm/kustomize/jq/yq/gh/vault/sops/age/
  k9s/terraform/go URLs, tarball subdirs (helm linux-${arch}, gh dir),
  and checksums are all arch-resolved with both arches pinned. Stamps
  and the Go tree are arch-specific, and an active-arch marker forces a
  republish of the single-arch ${bin} binaries when the pod moves
  between arches on the shared volume. Single fetch/verify helper kept.

Tests updated to assert the arch-aware form (both arches' Go checksums,
${dl_arch} templating) instead of the arm64-only literal.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-25 17:37:57 -03:00
jenkins
f8628e6ee0 Revert "hermes(agent): make titan-22 the strong primary home (no worker label)"
This reverts commit 2de52ec3e2e42696ae411482cb16f27ff1f5d273.
2026-08-25 16:54:04 -03:00
jenkins
5631366c4e hermes(agent): make titan-22 the strong primary home (no worker label)
APPLY ONLY AFTER the multi-arch hermes-agent image is built + validated
(both arch leaves + promoted index). Supersedes the earlier titan-22
flip on feature/hermes-agent-multiarch (dfa50b75), which required
worker=true and tolerated the old media-primary taint.

Rewrites the runtime node affinity so hermes-agent runs on titan-22:
- Adds a second, OR'd nodeSelectorTerm matching amd64 + hostname
  titan-22 + node-role.kubernetes.io/accelerator=true. It does NOT
  require node-role.kubernetes.io/worker (titan-22 is no longer a
  generic worker).
- Keeps the arm64 pi-fleet term untouched as an OR'd fallback so the
  worker is never stranded if titan-22 is unavailable.
- Makes titan-22 the STRONG/primary preference: a hostname=titan-22
  preference at weight 100 (the scheduler maximum) outranks the pi-fleet
  rpi5 nudge, lowered to weight 50, so hermes actually lives on titan-22.
- Tolerates node-role.kubernetes.io/accelerator=true:NoSchedule so it
  can consider titan-22; this does not change jellyfin's media-core
  priority or preemption.

Updates test_hermes_agent_layout.py to the two-term topology, the
[100, 50] preference weights, the titan-22 primary preference, the
absence of a worker requirement on the titan-22 term, and the toleration.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-25 16:43:32 -03:00
jenkins
8a71084585 build(hermes-agent): source base image + test deps from in-cluster mirrors
The hermes-agent-image pipeline failed intermittently on external network:
Kaniko's docker.io fallback for the base image is IPv6-broken from build
pods, and the "Validate reviewed release source" stage pip-installed pytest
from files.pythonhosted.org (DNS failures). Neither should touch the public
internet.

Base image: repoint the Dockerfile FROM from docker.io to the in-cluster
Harbor "mirror" project, keeping the exact content-addressed index digest
(9c841866...) and both arch leaves. A Flux-managed one-shot Job
(services/harbor/hermes-agent-base-mirror-job.yaml, suspend: true like the
cassandra bootstrap job) runs `skopeo copy --all` from docker.io into Harbor
using the same Vault-injected admin credential as the existing Harbor
immutability jobs; a tiny fail-closed helper ensures the public target
project first. Digest pinning and multi-arch are preserved; Kaniko pulls it
over the internal insecure registry with no docker.io fallback.

Test deps: install pytest/PyYAML fully offline (`pip --no-index
--find-links`) from a reviewed in-repo wheelhouse
(ci/vendor/hermes-agent-test-wheels) matching the arm64 python:3.12 build
container, so the validate stage never resolves a public index.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-25 13:53:22 -03:00
jenkins
2bfdee6169 fix(hermes-agent): do NOT make titan-24 a general worker for the amd64 build
titan-24 is an accelerator node (co-hosts the out-of-cluster Sui validator), not
a general worker. The amd64 build leg was requiring node-role worker=true, which
forced labeling titan-24 as a worker and opened it to unrelated cluster
scheduling. It already pins by hostname+arch, so drop the worker requirement and
remove the titan-24 worker-join from the node-prefer CronJob entirely. The build
targets titan-24 specifically (hostname) and tolerates its taint; nothing else
in the cluster gets scheduled there.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-25 11:50:52 -03:00
jenkins
ff2c003b4b build(hermes-agent): multi-arch image via two native kaniko legs
Repoint the hermes-agent base FROM at the upstream multi-arch OCI INDEX
digest (tag v2026.7.7.2, revision 9de9c25f) whose arm64 leaf is byte-for-byte
the previously pinned single-arch base, so the arm64 build is unchanged while
the same reviewed version now also resolves an amd64 leaf. Kaniko selects the
matching leaf per build platform.

Rework the release pipeline to build both arches natively and promote a
multi-arch image without switching off kaniko or weakening any existing
security assertion:

- Keep the arm64 kaniko leg on the unchanged rpi5 coordinating pod; it now
  pushes an arch-suffixed candidate tag (...-build-<N>-arm64).
- Add a second native amd64 kaniko leg on a titan-24-pinned, tolerating,
  resource-capped pod (ceiling strictly below the arm64 leg) that
  independently re-verifies the reviewed revision and stashes its leaf
  evidence (...-build-<N>-amd64).
- Add ci/scripts/hermes_multiarch_combine.py: a pure-python, fail-closed
  combiner that re-reads each per-arch leaf from the registry, proves its
  digest AND its config architecture, assembles a Docker manifest LIST
  (already inside the promote allow-list), refuses to overwrite an existing
  final tag, publishes the arch-less ...-build-<N> tag, and re-verifies the
  registry resolved the exact index referencing exactly the two leaves. It
  emits the index digest in the SAME digest-file/image-file format the
  single-arch step produced, so render/verify-evidence/hermes_oci_promote.py
  promote the INDEX with no change to those scripts.

Tests: add test_hermes_multiarch_combine.py (full hash/verification chain);
strengthen the image-builder suites for the two-arch topology (both kaniko
legs carry the reviewed heredoc-compat build-arg; amd64 leg pinned+capped+
boundary-checked; combine stage wiring; expanded evidence archive) without
weakening the arm64-leg assertions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-25 11:05:29 -03:00
jenkins
5b1f832072 hermes(voice): keep one consistent ellipsis on conversation status labels
The overlay status caption already carries a static ellipsis (e.g. 'Thinking…'),
and after ~2.5s of silent thinking the 'working' affordance added an animated
dots ::after on top of it — so the label intermittently rendered as 'Thinking……'
only on longer turns. Drop the animated text dots and keep the orb-halo shimmer
as the liveness cue, so the ellipsis stays a single, consistent static one.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-25 01:18:24 -03:00
jenkins
a071d091b0 hermes(voice): add a centered Start conversation button to the new-chat screen
Conversation mode was only reachable via a small icon by the composer, which is
easy to miss on a fresh session. Inject a prominent, centred 'Start
conversation' button into the empty new-chat state (#emptyState), below the
subtitle and above the suggestions, so it sits in the vertical centre of the
screen. It is created only when local voice is available, honours the same
show/hide preference as the composer toggle, and enters conversation mode
through the same activate() path. Fully guarded so it degrades to nothing if the
empty state is absent. New probe scenario verifies it is created, visible, and
opens the conversation overlay on press.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-25 00:13:11 -03:00
jenkins
ffa477585d hermes(webui): fix mobile composer hidden behind the gesture bar
On an Android standalone PWA the composer's bottom control row was rendering
about one line below the visible viewport, behind the system gesture bar, so
those controls were unreachable. The theme already pads the titlebar with
env(safe-area-inset-top/left/right), but the viewport meta never opted into
viewport-fit=cover, so every safe-area inset collapsed to 0 and the bottom edge
had no reservation.

Add viewport-fit=cover to the viewport meta (base patch) so the insets carry
real values, and reserve env(safe-area-inset-bottom) at the bottom of the
composer (brand.css), additive with the app's existing --keyboard-bottom-inset
and absorbed by the flex-1 message scroller so total height stays within the
viewport. Inert on desktop (env() resolves to 0). Dockerfile verifies the meta
patch landed; brand/dockerfile contracts covered by tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 23:32:27 -03:00
jenkins
fd0a4b23f9 hermes(voice): auto-retry transient provider errors; prime STT acronyms
Diagnosed from a live voice session (e1b9fccb90ef): three turns failed with
raw '**Error:** HTTP 502 ... hermes-{claude,codex}-broker' because the agent
pod hosting the model brokers rolled mid-conversation. The voice client
correctly refused to speak the error envelope, but it then dropped the user's
utterance and forced them to repeat it three times.

Voice: on a TRANSIENT provider error (5xx/502/'error sending request'/timeout),
conversation mode now re-runs the errored turn in place through the app's own
regenerate action (which truncates the errored turn — no duplicate user
message) and stays in Thinking so its cues cover the reconnect gap. Bounded to
MAX_TRANSIENT_RETRIES (2); a non-transient error or an exhausted budget still
drops cleanly to 'let's try that again — listening'. The raw error is never
spoken. New probe scenarios cover retry-then-recover and the bounded-then-drop
path; source contract updated.

STT: the same session mis-transcribed 'CUI' as 'cue'. Prime the default
initial_prompt with the domain acronyms the user uses (CUI, FOUO, DoD, NIST,
CMMC, FIPS, RMF, POA&M, ATO, SBU) so they bias to uppercase forms.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 22:10:21 -03:00
jenkins
12ec6e5a70 hermes(voice): crop Hermes character into orb as a feathered watermark
The orb watermark was showing the character art as a low-opacity square
box (visible edges) — the earlier screen-blend pass washed it into a glow
that lost the artwork. Reprocess the source into a feathered circle so the
orb crops it (no box) while keeping the character's facial features, paint
it with a normal blend at 0.5 opacity, and size it to fill the orb (inset
7%). Verified by compositing over a simulated orb before shipping.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 20:36:51 -03:00
jenkins
1baab8e014 hermes(voice): restore always-opening overlay, character watermark, speaker default
- REGRESSION FIX: the full-screen conversation overlay stopped opening
  (fell back to the inline bar) because building the language/output
  selectors inside the overlay's single try/catch could throw on a
  phone (navigator.mediaDevices/setSinkId). Overlay build is now two
  phases: the essential orb+captions+controls attach first; the
  selectors attach after, each guarded, so a selector failure omits only
  that control and never the visualization. Control builders can no
  longer throw (inert hidden fallback); device enumeration is async
  after attach. Regression test covers mediaDevices-undefined and
  enumerate-rejects.
- Orb watermark is the processed Hermes character glyph (feathered,
  circle-cropped, screen-blended so the face glows on the dark orb), no
  more white/grey box.
- Output selector defaults to the loudspeaker (excludes the
  communications/earpiece endpoint that the open mic otherwise forces);
  explicit choice sticks; hidden gracefully where setSinkId is
  unsupported. 291 voice tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 20:24:31 -03:00
jenkins
546e960185 hermes(stt-client): retry a brief STT outage, never dump a traceback
A transcription that landed while the private Whisper service was
restarting (an image roll) crashed hermes_stt_client.py with a raw
urllib ConnectionRefused traceback that got dumped into the
conversation. The client now retries the request with backoff (up to 5
attempts, ~10s - long enough to ride an STT pod restart) and, on a
persistent outage, exits with one concise line ('speech transcription
unavailable...') instead of a stack trace. Delivered via the coordinator
ConfigMap; the next reconcile picks it up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 20:16:57 -03:00
jenkins
4d93ef5a0e hermes(voice): workspace nav home, character orb, conversation rename, voice-lang fix
Final conversation-mode polish from mobile testing:
- The Workspace toggle now sits with the chat/Telegram nav at every
  width: nav.rail on desktop, the top app titlebar on mobile. The
  floating pill that pushed the mobile composer's control row (and the
  conversation-mode button) off screen is gone - a fallback exists only
  for headless DOMs and is pinned to a top corner, never over the
  composer.
- The conversation orb watermark is now the Hermes character avatar
  (static/hermes-agent-192.png) instead of the caduceus staff.
- User-facing 'hands-free' copy renamed to 'Conversation mode'.
- Wrong-voice fix: strongReplyLanguage flagged Spanish on a single
  accented char, so an English reply naming European cities (Zürich,
  Málaga) overrode the correct English STT detection and was spoken by
  the Spanish voice. Detection now requires density (Cyrillic >=4 at
  >=50%, or inverted punctuation / >=2 accents corroborated by Spanish
  stopwords); plain English always speaks English, forced language wins,
  accent-free Spanish still routes via trusted STT. 294 voice tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 19:22:32 -03:00
jenkins
181a7517c2 hermes(voice): interim-ack tail, natural fillers, unified audio + output picker
Some checks failed
Tests / Declarative: Post Actions failed: 49, skipped: 72, passed: 3724
Three conversation-mode fixes in one pass:
- Interim-acknowledgement truncation: when an interim ack folds into the
  hidden worklog segment mid-speech, the retained unspoken tail is now
  flushed and spoken in full before the Thinking transition, and a
  distinct follow-up message is chunked from its own start and queued
  after the interim drains (no more 'stops after the first clause, rest
  resurfaces with the next message').
- Natural thinking fillers: brief per-language interjections (Umm/Hmm/
  One sec; Mmm/A ver; Хм/Секунду) on genuine >1.9s thinking gaps only,
  non-repeating, answer-preempting, mute-aware.
- One unified audio sink for every spoken output (reply, cues, fillers,
  WAV fallback) - fixes cues playing the loudspeaker while the reply
  used a different output - plus a tidy corner output-device selector
  (enumerateDevices + setSinkId, feature-detected, session-only) styled
  like the language selector. Also realigns two STT-server decode-param
  assertions to the dict form from the STT tuning commit. 284 voice tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 17:32:13 -03:00
jenkins
64b7bc55cf hermes(stt): tune final decode for fast-and-accurate (beam 2, name priming)
Default final beam 5 -> 2: the accuracy/speed knee - most of beam
search's benefit at ~2x greedy instead of ~5x, protecting commit
latency on the Jetson (still env-overridable via
HERMES_STT_FINAL_BEAM_SIZE). Prompt now also primes common names (Amy,
Claude, Hermes) so 'Amy' stops transcribing as 'aiming'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 16:57:50 -03:00
jenkins
1fdfaff096 hermes(stt): accurate large-v3-turbo final decode, fast tiny partials
Proper nouns (Córdoba, Cancún) and dropped words came from decoding the
committed transcript with the small model. The image already ships
large-v3-turbo, so the final decode now uses it with beam_size=5, a
temperature fallback ladder, and a proper-noun/accents initial_prompt
that fixes first-pass capitalization and diacritics across EN/ES/RU;
the rolling previews stay on tiny at greedy so the on-the-fly feel is
unchanged. The accurate decode runs in the speculative predecode during
the end-of-speech silence and is cache-reused at commit, so perceived
latency stays low. All decode knobs are env-overridable for on-device
tuning (beam/temperature/prompt), with small as the guaranteed-present
rollback if turbo underperforms on the Jetson. 206 STT tests pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 16:39:00 -03:00
jenkins
f7b3506a88 hermes(chat): parse large cluster reads before capping the output
The size cap was applied to the raw wire body, so a nodes list (huge
because of status.images) truncated before the image-stripping ran and
came back as a truncation notice. Accept up to 6 MiB on the wire to
parse and clean, then enforce the 384 KiB model-facing cap on the
stripped result - nodes now returns real data. Delivered via the
cluster-read ConfigMap; picked up on the next pod roll.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 16:25:21 -03:00
jenkins
b40300efaf hermes(voice): a stray error segment must not discard a real reply
The 'Something went wrong - listening' state with no spoken answer was a
false positive: readAssistantTurn flagged the whole turn as an error if
ANY segment was error-stamped - including a recovered/transient tool
error or a cancellation notice from an earlier interim - and threw away
the real answer that the same turn produced. Error now surfaces only
when the turn yielded no spoken answer at all; a turn with real content
is spoken normally. Softened the genuine-error label to the friendlier
'Let's try that again - listening'. New probe scenarios lock both: an
error segment alongside an answer speaks the answer (error=false), and
an error-only turn still reports the error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 16:18:48 -03:00
jenkins
3fd760de0e hermes(voice): fix HHermesProcessed + first-sentence-stop; orb mark, lang selector
Real root cause (confirmed against the live build-24 DOM): the caption
and TTS extraction fell back to turn.textContent whenever a settle-frame
race left no readable answer segment, scraping the avatar letter,
author name and 'Processed 13s' chip - and that truncated reply made
TTS speak only the first segment then drop to Listening even with the
mic muted (the muted-mic first-sentence-stop). Extraction now prefers
each answer segment's data-raw-text, else the answer .msg-body only
(excluding thinking/tool/worklog/role chrome), and the textContent
fallback is gone; a genuinely mid-flight reply retries briefly so the
whole thing is read before the overlay drains. Also: the app's own
caduceus mark embedded in the conversation orb as a subtle watermark; a
corner language selector (Auto + en/es/ru) that forces both the STT
hint and the reply voice; and a thinking affordance after 2.5s of dead
time. New response probe + extraction test prove caption==body-only and
that every sentence reaches the TTS queue. 278 voice-lane tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 15:46:09 -03:00
jenkins
13a058a5c8 hermes(chat): teach the assistant cluster_read and voice acknowledgements
The chat prompt still forbade all cluster access, so Hermes told users
it had no visibility even though cluster_read is live - it now knows it
has a read-only cluster tool (no Secrets/Vault) and should use it rather
than deny. Adds tool/research-conditioned acknowledgement guidance: when
a turn needs a lookup, plan or calculation, open with one short 'on it,
~ETA' line then deliver the full answer; simple questions get no
preamble. In spoken mode that first line is read aloud. Fixes two
pre-existing exact-match test pins the HUX/cluster rollout had grown
(plugins.enabled list; a fieldRef env comprehension).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 15:20:42 -03:00
jenkins
ed0bc30d7c hermes(chat): make cluster_read robust to large node listings
Node .status.images (every cached image on the node) overflowed the
size cap and left json.loads parsing a truncated blob, so a nodes query
came back as a non-JSON error. The de-noise pass now summarizes that
list, and genuine truncation returns the readable prefix with a
narrow-your-query hint instead of an error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 14:40:09 -03:00
jenkins
c4eb872690 monitoring(ai): show Claude Fable weekly quota 2026-08-24 14:13:35 -03:00
jenkins
d041f1d1ee hermes(voice): round-3 fixes, language routing, session-toast root cause
- Captions read message bodies only (the scraper was concatenating
  avatar, author and worklog chips); multi-segment interim turns are
  now speakable and drive clean speak-to-thinking-to-speak cycles when
  playback drains mid-turn.
- Dynamic endpointing: complete-looking partials (3+ words or terminal
  punctuation) endpoint at the base window; the long hold remains only
  for one-two-word fragments. A stale-busy 10s settle wait on every
  post-error send is gone.
- Both overlay captions are bounded, touch-scrollable regions with
  follow-tail; caps raised for long turns.
- Error envelopes are never spoken or captioned; errored turns run
  resyncCapture (fresh STT session on the hot mic).
- Workspace toggle now lives in the sidebar rail (floating button only
  below the rail breakpoint).
- False 'session unavailable' toast root-caused: the router continuity
  guard shows it on a 409 that fired when a transient profile-listing
  failure failed closed into a fake cross-profile mismatch; the patcher
  now answers from the alias cache and never claims a default-vs-named
  mismatch while aliases are unconfirmed.
- Barge-in sends carry a one-line cut-point marker with the last spoken
  sentence; visible-history truncation judged infeasible client-side.
- Language switching works end-to-end: sticky per-session STT language
  hint (restarting an unused next session on switch), reply voice from
  script evidence, STT detection, then stopword heuristic; cues and WAV
  fallback share the turn language.
- Legacy CI guards: node skip for the DOM probe, ffmpeg/codec skips for
  the container-fallback test. 272 voice-lane tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 13:58:35 -03:00
jenkins
c0a9c92ee4 hermes(worker): stage inert HUX foundation on the worker instance
worker.bstein.dev (the hermes-agent Deployment) gains the same HUX
shape as chat, staged and inert: a foundation-only hux sidecar on the
reviewed WebUI image line (Flux setters bound, 5s probe budgets), an
init that provisions the HMAC identity as slot-100 on the durable home
subtree (create-once context key, O_EXCL subject binding, per-pod
worker key; no relay/router/evidence keys so those trusts fail closed),
and observe-only hook env in the agent container with the runtime
plugin mounted but deliberately NOT enabled - activation is a reviewed
one-line flip per docs/hux/WORKER-PLAN.md, which carries the rollout,
verification gates, canary/rollback ladder and open questions.
Cross-surface continuity remains unclaimed until the live gates pass.
7 new topology-adaptive delivery gates green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 13:54:27 -03:00
jenkins
3bca7b7465 hermes(hux): inline approval prompts and ask-not-deny defaults
Re-enforcement prerequisites, code-side complete: the autonomy runtime
now docks pending-approval cards above the composer (newest first, cap
three, aria-live, allow-once / always / deny wired to the existing
decide route with idempotency; polling gated to active turns and
fail-tolerant), so parked tool calls are never silent. The default
capability matrix no longer denies by default: network and web_search
ask below autonomous (visible prompt) and nothing resolves to deny
except explicit grants or private mode; SO-39 stays intact - deploy and
external side effects always ask and external never auto-allows.
rules.py at 100% line+branch; 744 hux-lane tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 13:53:00 -03:00
jenkins
d1225bcab4 hermes(chat): read-only Atlas cluster visibility for chat
RBAC: the built-in view ClusterRole (which never includes Secrets, so
Vault-managed material stays structurally invisible) plus a read-only
extra for nodes, namespaces, PVs, storage classes, CRDs, Flux resources
and metrics, bound to the chat service account. Tooling: a cluster-read
plugin registers a GET-only cluster_read tool against the in-cluster
API using the pod's projected token - secrets paths refused in the
handler as well, malformed segments rejected, responses bounded and
stripped of managedFields noise. Classified read_files/low in the HUX
capability map. RBAC applies on push; the tool activates when the pods
next roll (bundled with the round-3 voice build).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 13:33:48 -03:00
jenkins
043aa9ee89 release(hermes): bind block-style HUX build metadata
The env setters moved to block style so Flux can rewrite them; the
renderer belt now matches the same shape and the release test asserts
the bound value lines rather than the old flow mapping.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 11:21:35 -03:00
jenkins
09ba5f8ac0 hermes(voice): fix first-word clipping, always-stitch, panel glow, conversation mode
Round 2 from live testing:
- Clipping root cause: the endpointer committed on any 1.1s pause, so a
  thinking pause after a sentence opener sent one word; a speculative
  Whisper pass over that fragment then stalled the real commit ~3.5s on
  the Jetson. Young utterances now hold a 1.8s endpoint until 1.2s of
  speech accrues, speculative decode waits for 700ms of speech, resume
  is unconditional after any 650ms gap (server-frozen snapshots can
  never reach commit), and the noise floor is capped so playback echo
  cannot deafen onset. Deterministic capture-continuity probe added.
- Stitching now fires for any barge-cancelled send within 20s,
  regardless of partial assistant output.
- The workspace-drawer dark rectangle was our own HUX chrome resolving
  undefined theme tokens (--bg-primary) to a flat box; bootstrap.css
  bridges the real theme tokens and drops a compositor-hazard
  backdrop-filter.
- Conversation mode: full-viewport hands-free overlay with an energy-
  driven orb (mic RMS in, TTS activity out), state caption, live
  transcript/reply captions, mute and exit controls, focus trap,
  Escape, reduced-motion support. Voice lane 253 tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 11:21:35 -03:00
jenkins
1793903f5e hermes(chat): observe-only enforcement and honest tool capabilities
Real traffic showed the first enforcement pass blocking core assistant
faculties: skills listing and the sandboxed Python classified as
unknown external side effects, browsing denied by default, with no
approval surface in the chat flow. Enforcement returns to observe-only
fleet-wide while the approvals UX and default grants are reworked, and
the capability map now tells the truth about the real toolset: the
Python sandbox is internal shell work, skills/todo/clarify/vision are
reads, browsing is network (medium), image generation writes an
artifact through the trusted broker. Unknown tools remain fail-closed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 10:46:54 -03:00
jenkins
7d070c4e1c test(hermes): skip node-dependent HUX suites on node-less runners
The titan-iac CI pod has no node binary; the gate helper and every
direct node invocation in the new HUX suites now skip with an explicit
reason instead of erroring, restoring the main-CI baseline. Runners
with node keep full enforcement.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 06:32:46 -03:00
jenkins
91cfb901a2 hermes(voice): continuous mic, barge stitching, 1.15x speech
Three conversational fixes for hands-free chat:
- The microphone now stays hot for the whole session: capture runs on
  its own epoch, re-arms immediately after each utterance endpoints,
  and keeps recording through transcribing/thinking/speaking - speech
  is never lost to Hermes being busy. Speech onset during a response
  cancels it through the live capture path (echo-guarded exactly like
  the old monitor) without touching the running recorder.
- When the user talks over Hermes before any visible reply appeared,
  the interrupted utterance and the follow-up are stitched into one
  message (20s window), so the response addresses the whole thought.
- TTS speaks 15% faster by default (server-side length_scale, no pitch
  shift), user-tunable via hermes-voice-tts-speed (0.5-2.0), honored on
  streaming, WAV fallback and thinking-cue paths.

245 voice-lane tests pass; single getUserMedia site preserved;
Dockerfile grep guards verified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 06:27:38 -03:00
jenkins
17037773bf test(hermes): delivery gate accepts the enforced HUX posture
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 06:22:26 -03:00
jenkins
39a544828f hermes(chat): activate HUX foundation canary on ordinal 3
Some checks failed
Tests / Declarative: Post Actions failed: 71, skipped: 28, passed: 3680
Re-applies the staged HUX topology pinned to the reviewed build-21
image (git-2f535d3a...-build-21-release@sha256:e5b9b2fa...), with the
first-activation posture: HUX_FLAGS=hux.foundation only,
HUX_TOOL_ENFORCEMENT=0, and a RollingUpdate partition of 3 so only
hermes-chat-tenant-3 rolls. Adds the /healthz auth bypass on the chat
proxy so HUX-12 health receipts can observe a real 200, points the
evidence policy at it, and makes the delivery flag gate progressive
(foundation first, cards enabled per lifecycle acceptance; unknown
flags still never ship).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 05:47:46 -03:00
jenkins
2f535d3a30 test(hermes): portable node coverage gate for the HUX suites
Build 20 failed on the CI image's Node 20: --test-coverage-lines and
friends need Node >= 22.8 and --experimental-strip-types needs 22.6.
A shared helper now runs plain --experimental-test-coverage and
enforces the same per-source >=95 floors by parsing the coverage
table, so the gate is identical on Node 20 and newer local Nodes; the
TypeScript suites skip with an explicit reason on runtimes that cannot
strip types. Per-file gating also exposed pre-existing debt the old
aggregate thresholds hid (wave_b_projects_modes.js branches 90 / funcs
94.7) - recorded as explicit enforced floors, not waived.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 05:22:01 -03:00
jenkins
ffa003d3fb test(hermes): tolerate fieldRef env entries in chat voice gate
The webui container now carries a POD_NAME fieldRef for HUX subPathExpr
mounts; the voice routing assertions only ever inspected literal
values.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 04:45:15 -03:00
jenkins
dc034cb738 hermes(hux): bounded message-text search with privacy enforcement
HUX-03: GET /hux/v1/search now accepts include=message_text, an
explicit opt-in that scans the stored message events of the 100 most
recently active candidate conversations. Forgotten (tombstoned)
conversations, private-mode conversations, restricted events and fully
redacted events never match; the default indexed-fields search and its
response contract are unchanged (the shipped UI keeps requiring
message_text in not_indexed). Paginated mode uses one deterministic
total order (score desc, updated_at desc, id) with an offset cursor.
organization.py at 98% branch coverage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 04:43:08 -03:00
jenkins
e40fc5ec5d hermes(chat): stage the HUX-12 evidence producer sidecar
Activation-layer staging, fail-closed until enablement: a per-tenant
hux-evidence-producer sidecar on the exact reviewed WebUI image runs
hux_producer.run_once on a 60s loop, inert until the Vault-staged
evidence key (tolerant init, tmpfs, 0400, staged only for the hux
service and producer containers - never hermes or webui), the policy
ConfigMap, and the scope ConfigMap exist. Adds least-privilege
read-only RBAC (pods+statefulset in hermes, the single named Flux
Kustomization), tenant egress to the Kubernetes API ClusterIP and the
traefik edge, the policy allowlist, hux_producer packaging in the WebUI
image, and a third expected WebUI consumer in the Flux release
renderer. Delivery and image-automation gates enforce the boundary.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 04:39:45 -03:00
jenkins
469e52fd20 hermes(hux): add the HUX-12 release evidence producer
A companion package (outside the network-free hux/ service package)
that independently verifies and binds the whole release chain before
any transition: reviewed proposal URL, Jenkins job/build/result and
revision, immutable Harbor tag/digest equality, Flux kustomization and
applied revision with pin containment, desired workload image, every
Ready pod imageID, bounded-age health receipt, and rollback target.
Pure injectable verifier core, HTTPS-only collectors (SA token for the
Kubernetes API), and an evidence-trust driver that posts exactly one
If-Match transition with deterministic idempotency. Rejects stale,
replayed, downgraded, incomplete, cross-workload, mismatched, and
self-asserted evidence. 100% line and branch coverage (71 tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
2026-08-24 04:35:12 -03:00