titan/atlas-iac
3
0
Fork 0
Code Issues Pull Requests 9 Actions Packages Projects Releases Wiki Activity
atlas-iac/services/hermes/skills/master-hermes-on-atlas/references/architecture.md
Hermes Agent 94106bf252 refactor(hermes): rename the owner agent host to worker.bstein.dev 
Introduce worker.bstein.dev as the canonical hostname for the owner-only
Hermes coordinator, previously agent.hermes.bstein.dev.

The rename is additive, matching the shape #38 restored for chat and triage.
CoreDNS, both agent Ingresses and the hermes-sites certificate now serve BOTH
names, so merging this cannot take away the endpoint the operator uses to
reach the coordinator. Retiring agent.hermes.bstein.dev is a separate,
separately scheduled change. No redirect middleware is added.

What switches to the new host:
- HERMES_DASHBOARD_PUBLIC_URL and the oauth2-proxy --redirect-url
- the Keycloak agent proxy rootUrl
- operator docs, skills, the ZAP baseline target and the triage monitor default

What stays dual-homed until retirement:
- CoreDNS hosts entry, both agent Ingress rules, certificate SANs
- API_SERVER_CORS_ORIGINS (now a comma-separated pair)
- the Keycloak redirect URIs, web origins and post-logout origins, so a
  rollback only needs the oauth2-proxy --redirect-url reverted and does not
  require re-running the ensure job

The agent client passes its legacy origin through the optional fourth argument
#38 added to ensure_proxy_client, so no second mechanism is introduced. The
immutable ensure Job goes -11 -> -12 because #38 already consumed -11 and that
run has completed; without a further bump this change would never be applied.
Login on the new host fails until the -12 Job completes.

Because the session and CSRF cookies use the __Host- prefix they are bound to
one origin, so a fresh login must start on worker.bstein.dev and existing
sessions do not carry over -- re-login is required after rollout.

#38's public-host continuity test now covers the agent proxy's dual origins
rather than asserting the agent surface was untouched by the rename.

Knowledge catalogs and diagrams regenerated with `make knowledge`.
2026-08-21 10:29:46 +00:00

5.8 KiB
Raw Blame History

Live Atlas Hermes architecture

Use this reference for deployment-specific facts. Re-check live state before asserting health, placement, ownership, or current model availability.

Two isolated agent instances

Surface Purpose Identity boundary State and permissions
triage.bstein.dev Brad's automated testing triage Keycloak plus an outer oauth2-proxy exact-email allow-list for brad@bstein.dev hermes namespace, its own PVC and service account; read-only cluster triage plus approved internal evidence endpoints
worker.bstein.dev Brad's project coordinator Keycloak plus an outer oauth2-proxy exact-email allow-list for brad@bstein.dev hermes namespace and separate PVC; native Hermes delegates bounded work while Herdr preserves real Codex and Claude Code CLI sessions when needed
chat.bstein.dev Private consumer chat and research through Hermes WebUI or a linked Telegram DM Keycloak login plus one-time Telegram account link One Hermes process and PVC per assigned Keycloak subject; no Kubernetes RBAC, terminal, or private-service access

The instances do not share conversation state, credentials, profiles, skills created on their PVCs, or Kubernetes identities. They share only the inference service described below. Consumer users cannot reach the operator dashboard through the chat Service or Ingress.

Inference path

operator Hermes ───> independently authenticated openai-codex/gpt-5.6-terra
consumer Hermes ───> independently authenticated openai-codex/gpt-5.6-terra
       │
       └─ provider error or manual selection ─> hermes-model-gate
                                                │
                                                ├─ Hermes owns GPU ─> Ollama gpt-oss:20b
                                                └─ Wolf owns GPU ───> HTTP 503
  • The configured context length is 64,000 tokens.
  • hermes-model-gate is the stable OpenAI-compatible endpoint.
  • A coordination Lease stores whether Hermes or Wolf/Moonlight owns the GPU. Ariadne has narrowly scoped permission to update that Lease during handoff.
  • Wolf ownership makes the local gate return a deliberate unavailable response. Normal interactive chat remains on its Codex primary and does not consume titan-24 GPU resources.
  • The two agent pods do not consume titan-24 GPU memory. Ollama does. Agent pods are ARM gateway workloads with persistent state on separate PVCs.
  • Codex credentials are deliberately per-instance. Never copy Brad's Codex credential store into the consumer PVC.

Verify rather than memorize:

hermes status
hermes fallback list
kubectl -n hermes get lease titan-24-gpu-owner -o yaml
kubectl -n hermes get deploy hermes hermes-model-gate hermes-ollama -o wide
kubectl -n hermes-chat get deploy hermes-chat -o wide

Do not change the GPU owner as part of a lesson.

Retained OpenClaw replacement evidence

OpenClaw is not part of the current inference path, but its deployment history is retained:

  • 1bc58e10 added the OpenClaw testing-triage workspace, workload, model, ingress, RBAC, and storage.
  • follow-up commits wired local diagnosis, OIDC, persistence, and the gateway.
  • 97ecb36b replaced OpenClaw with Hermes and moved the agent-facing resources.
  • bb5e286f decoupled Hermes from the retired OpenClaw Flux dependency.
  • the openclaw-home PVC remains bound as retained state while no OpenClaw workload is running.

Have Brad verify the commits and live resource shape. This establishes the cluster deployment and replacement sequence; Brad's direct experience remains the evidence for how he personally used the old agent.

Operator evidence path

The operator role is designed for supervised test-failure triage:

  1. Ariadne deterministic bundle and optional diagnosis.
  2. Retained Jenkins logs and artifacts.
  3. Recent Git changes and Flux revision/state.
  4. Kubernetes workload, node, event, storage, DNS, and dependency health.
  5. Pushgateway quality metrics and VictoriaMetrics/Grafana context.
  6. A concise finding, confidence, evidence, blast radius, next checks, repo-side fix, and approval-required section.

The versioned $triage-titan-test-failures skill encodes this workflow. The operator has internal URLs for Ariadne, Jenkins, and VictoriaMetrics. The consumer instance intentionally does not.

Security layers and their limits

Security is layered rather than delegated to a prompt:

  1. Keycloak authenticates the human.
  2. The operator proxy limits agent.bstein.dev to Brad's exact email.
  3. Separate namespaces, PVCs, service accounts, configs, and ingresses isolate operator and consumer state.
  4. Kubernetes RBAC grants the consumer get/list/watch only and excludes Secrets, exec, attach, port-forward, and mutation.
  5. NetworkPolicy permits the consumer to reach DNS, the shared model gate, the Kubernetes API, and public IPv4 while blocking arbitrary private service access.
  6. Hermes deny patterns and instructions add user-facing guardrails.

Prompt rules and shell deny patterns are not the primary security boundary. RBAC, network policy, credential separation, and routing isolation are. The consumer can still disclose non-secret cluster metadata it is authorized to read; that is an intentional requirement and should be described honestly.

CLI-to-web mental model

The pinned Hermes build exposes chat, models and fallback, auth, sessions, profiles, skills and bundles, plugins, tools, MCP, gateway/channels, pairing, webhooks, cron, kanban, projects, hooks, memory/journey, logs, security, backup, and diagnostics. The web interface is a control surface over many of these same persisted capabilities.

Use <command> --help and the web Documentation page for the installed version. Do not rely on screenshots or online docs from a different release when a live command can settle the question.