Now that both the agent image (a68d1c4d, via the kustomize images: override)
and the hux sidecar (build-39) are multi-arch with amd64 leaves, move the worker
onto the amd64 accelerator titan-22:
- Add an OR'd nodeSelectorTerm for amd64 + node-role.kubernetes.io/accelerator +
hostname titan-22, with NO worker=true requirement. Keep the arm64 pi-fleet
term as an OR'd fallback so the worker is never stranded.
- Strong primary preference: hostname=titan-22 at weight 100 (scheduler max),
pi-fleet rpi5 nudge lowered to 50, so hermes actually lives on titan-22.
- Tolerate node-role.kubernetes.io/accelerator=true:NoSchedule (harmless where
absent) and the soft atlas.bstein.dev/media-primary:PreferNoSchedule that
titan-22 currently carries, so the weight-100 preference is not offset and
placement is deterministic.
Completes the titan-22 effort the flip branches missed; the earlier branches
never repointed to multi-arch images, which is why the worker never landed here.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
The webui release handoff verified org.opencontainers.image.revision on the
Harbor artifact's own extra_attrs.config.Labels. That works for a single-arch
image, but a multi-arch manifest list has no top-level config, so Harbor reports
the label on each per-arch child. build-38 built + published the index fine, then
failed post-publish with 'Harbor artifact omitted OCI image labels'.
verify_registry_digest now checks the top-level config labels when present
(single-arch, unchanged) and otherwise walks the index references, fetching each
child artifact by digest and asserting its revision label. Mirrors how the agent
image lane already tolerates a multi-arch index, without dropping the supply-chain
label check. Adds multi-arch pass/reject tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
test_gateway_image_honors_ui_model_and_caps_reasoning still pinned the old
arm64-only docker.io base (nousresearch/hermes-agent@sha256:47d4bd4c...). The
agent image moved to the multi-arch mirror base
(harbor-core.harbor.svc.cluster.local/mirror/hermes-agent@sha256:9c841866...)
in 8a710845 for the two-leg build; the test wasn't updated, so it was a latent
red only the webui/quality lane runs. Point the assertion at the current base.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Make registry.bstein.dev/bstein/hermes-webui a linux/amd64 + linux/arm64
manifest list so the agent pod's `hux` sidecar (which runs the webui image)
can schedule onto the amd64 node titan-22. Reuses the hermes-agent multi-arch
pattern already on main.
- Dockerfile.hermes-webui: repoint both FROMs to multi-arch, internal sources.
The upstream WebUI base (ghcr sha256:a83a3893..., already a multi-arch OCI
index) is now pulled from the in-cluster Harbor mirror; the agent base moves
from the retired arm64-only leaf (81970563) to the multi-arch agent index
(a68d1c4d). Kaniko selects the matching arch leaf per build node.
- services/harbor/hermes-webui-base-mirror-job.yaml: new suspended, operator-run
skopeo `copy --all` Job mirroring the upstream WebUI base index into Harbor's
`mirror` project (modeled on hermes-agent-base-mirror-job.yaml; reuses the
generic ensure-project helper). Wired into the harbor kustomization.
- Jenkinsfile.hermes-webui-image: arm64 leg (titan-20) + amd64 leg (titan-24,
hostname+arch pin, toleration Exists, resource-capped, own checkout scm) +
Combine multi-arch index stage; per-arch evidence archived alongside the index.
- hermes_multiarch_combine.py: generalize the destination pattern/component to
serve both hermes-agent and hermes-webui (fail-closed to just those two).
- Tests updated to the two-arch topology (two legs, combine, both FROM bases,
the mirror Job, twelve archived evidence files).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
The hermes-agent installs its CLI toolchain at runtime into the shared
/opt/data/tools Longhorn volume, but every download hardcoded arm64. On
the amd64 node titan-22 that left configure-agent-clients failing with
"Missing optional dependency @openai/codex-linux-x64" and the operator
toolchain fetching arm64 binaries, so the pod churned.
Detect the running node's arch (uname -m; fail closed on anything but
aarch64/x86_64) and resolve every asset per-arch:
- install-agent-tools init script (agent-deployment.yaml): ttyd and
kubectl download the arch-correct asset with the arch-correct sha256
(real ttyd 1.7.7 x86_64 and kubectl v1.33.3 amd64 checksums added; the
arm64 ones kept). The npm CLI stamp is now arch-specific
(.cli-versions-<vers>-${arch}) so a fresh arch re-runs npm install and
pulls its own native optional deps; npm keeps both arches' packages.
- install_agent_tools.sh: flux/helm/kustomize/jq/yq/gh/vault/sops/age/
k9s/terraform/go URLs, tarball subdirs (helm linux-${arch}, gh dir),
and checksums are all arch-resolved with both arches pinned. Stamps
and the Go tree are arch-specific, and an active-arch marker forces a
republish of the single-arch ${bin} binaries when the pod moves
between arches on the shared volume. Single fetch/verify helper kept.
Tests updated to assert the arch-aware form (both arches' Go checksums,
${dl_arch} templating) instead of the arm64-only literal.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
APPLY ONLY AFTER the multi-arch hermes-agent image is built + validated
(both arch leaves + promoted index). Supersedes the earlier titan-22
flip on feature/hermes-agent-multiarch (dfa50b75), which required
worker=true and tolerated the old media-primary taint.
Rewrites the runtime node affinity so hermes-agent runs on titan-22:
- Adds a second, OR'd nodeSelectorTerm matching amd64 + hostname
titan-22 + node-role.kubernetes.io/accelerator=true. It does NOT
require node-role.kubernetes.io/worker (titan-22 is no longer a
generic worker).
- Keeps the arm64 pi-fleet term untouched as an OR'd fallback so the
worker is never stranded if titan-22 is unavailable.
- Makes titan-22 the STRONG/primary preference: a hostname=titan-22
preference at weight 100 (the scheduler maximum) outranks the pi-fleet
rpi5 nudge, lowered to weight 50, so hermes actually lives on titan-22.
- Tolerates node-role.kubernetes.io/accelerator=true:NoSchedule so it
can consider titan-22; this does not change jellyfin's media-core
priority or preemption.
Updates test_hermes_agent_layout.py to the two-term topology, the
[100, 50] preference weights, the titan-22 primary preference, the
absence of a worker requirement on the titan-22 term, and the toleration.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
The hermes-agent-image pipeline failed intermittently on external network:
Kaniko's docker.io fallback for the base image is IPv6-broken from build
pods, and the "Validate reviewed release source" stage pip-installed pytest
from files.pythonhosted.org (DNS failures). Neither should touch the public
internet.
Base image: repoint the Dockerfile FROM from docker.io to the in-cluster
Harbor "mirror" project, keeping the exact content-addressed index digest
(9c841866...) and both arch leaves. A Flux-managed one-shot Job
(services/harbor/hermes-agent-base-mirror-job.yaml, suspend: true like the
cassandra bootstrap job) runs `skopeo copy --all` from docker.io into Harbor
using the same Vault-injected admin credential as the existing Harbor
immutability jobs; a tiny fail-closed helper ensures the public target
project first. Digest pinning and multi-arch are preserved; Kaniko pulls it
over the internal insecure registry with no docker.io fallback.
Test deps: install pytest/PyYAML fully offline (`pip --no-index
--find-links`) from a reviewed in-repo wheelhouse
(ci/vendor/hermes-agent-test-wheels) matching the arm64 python:3.12 build
container, so the validate stage never resolves a public index.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
titan-24 is an accelerator node (co-hosts the out-of-cluster Sui validator), not
a general worker. The amd64 build leg was requiring node-role worker=true, which
forced labeling titan-24 as a worker and opened it to unrelated cluster
scheduling. It already pins by hostname+arch, so drop the worker requirement and
remove the titan-24 worker-join from the node-prefer CronJob entirely. The build
targets titan-24 specifically (hostname) and tolerates its taint; nothing else
in the cluster gets scheduled there.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Repoint the hermes-agent base FROM at the upstream multi-arch OCI INDEX
digest (tag v2026.7.7.2, revision 9de9c25f) whose arm64 leaf is byte-for-byte
the previously pinned single-arch base, so the arm64 build is unchanged while
the same reviewed version now also resolves an amd64 leaf. Kaniko selects the
matching leaf per build platform.
Rework the release pipeline to build both arches natively and promote a
multi-arch image without switching off kaniko or weakening any existing
security assertion:
- Keep the arm64 kaniko leg on the unchanged rpi5 coordinating pod; it now
pushes an arch-suffixed candidate tag (...-build-<N>-arm64).
- Add a second native amd64 kaniko leg on a titan-24-pinned, tolerating,
resource-capped pod (ceiling strictly below the arm64 leg) that
independently re-verifies the reviewed revision and stashes its leaf
evidence (...-build-<N>-amd64).
- Add ci/scripts/hermes_multiarch_combine.py: a pure-python, fail-closed
combiner that re-reads each per-arch leaf from the registry, proves its
digest AND its config architecture, assembles a Docker manifest LIST
(already inside the promote allow-list), refuses to overwrite an existing
final tag, publishes the arch-less ...-build-<N> tag, and re-verifies the
registry resolved the exact index referencing exactly the two leaves. It
emits the index digest in the SAME digest-file/image-file format the
single-arch step produced, so render/verify-evidence/hermes_oci_promote.py
promote the INDEX with no change to those scripts.
Tests: add test_hermes_multiarch_combine.py (full hash/verification chain);
strengthen the image-builder suites for the two-arch topology (both kaniko
legs carry the reviewed heredoc-compat build-arg; amd64 leg pinned+capped+
boundary-checked; combine stage wiring; expanded evidence archive) without
weakening the arm64-leg assertions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
The overlay status caption already carries a static ellipsis (e.g. 'Thinking…'),
and after ~2.5s of silent thinking the 'working' affordance added an animated
dots ::after on top of it — so the label intermittently rendered as 'Thinking……'
only on longer turns. Drop the animated text dots and keep the orb-halo shimmer
as the liveness cue, so the ellipsis stays a single, consistent static one.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
Conversation mode was only reachable via a small icon by the composer, which is
easy to miss on a fresh session. Inject a prominent, centred 'Start
conversation' button into the empty new-chat state (#emptyState), below the
subtitle and above the suggestions, so it sits in the vertical centre of the
screen. It is created only when local voice is available, honours the same
show/hide preference as the composer toggle, and enters conversation mode
through the same activate() path. Fully guarded so it degrades to nothing if the
empty state is absent. New probe scenario verifies it is created, visible, and
opens the conversation overlay on press.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
On an Android standalone PWA the composer's bottom control row was rendering
about one line below the visible viewport, behind the system gesture bar, so
those controls were unreachable. The theme already pads the titlebar with
env(safe-area-inset-top/left/right), but the viewport meta never opted into
viewport-fit=cover, so every safe-area inset collapsed to 0 and the bottom edge
had no reservation.
Add viewport-fit=cover to the viewport meta (base patch) so the insets carry
real values, and reserve env(safe-area-inset-bottom) at the bottom of the
composer (brand.css), additive with the app's existing --keyboard-bottom-inset
and absorbed by the flex-1 message scroller so total height stays within the
viewport. Inert on desktop (env() resolves to 0). Dockerfile verifies the meta
patch landed; brand/dockerfile contracts covered by tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
