titan/atlas-iac
3
0
Fork 0
Code Issues Pull Requests 9 Actions Packages Projects Releases Wiki Activity
atlas-iac/services/hermes/skills/master-hermes-on-atlas/references/architecture.md
jenkins c87a4b5be5 ai(hermes): add operator mastery lab
2026-08-02 02:47:55 -03:00

4.4 KiB
Raw Blame History

Live Atlas Hermes architecture

Use this reference for deployment-specific facts. Re-check live state before asserting health, placement, ownership, or current model availability.

Two isolated agent instances

Surface Purpose Identity boundary State and permissions
agent.bstein.dev Brad's operator and triage lab Keycloak plus an outer oauth2-proxy exact-email allow-list for brad@bstein.dev hermes namespace, its own PVC and service account; read-only cluster triage plus approved internal evidence endpoints
chat.bstein.dev Consumer chat and research Keycloak login hermes-chat namespace, separate PVC/config/auth/sessions/skills; read-only cluster observer RBAC and no access to Secrets or Kubernetes mutation APIs

The instances do not share conversation state, credentials, profiles, skills created on their PVCs, or Kubernetes identities. They share only the inference service described below. Consumer users cannot reach the operator dashboard through the chat Service or Ingress.

Inference path

operator Hermes ─┐
                 ├─> hermes-model-gate ─> Ollama gpt-oss:20b on titan-24 GPU
consumer Hermes ─┘             │
                               └─ 503 while Wolf owns the GPU

each instance independently ─────> openai-codex/gpt-5.4 fallback
  • The configured context length is 64,000 tokens.
  • hermes-model-gate is the stable OpenAI-compatible endpoint.
  • A GPU-owner ConfigMap and controller coordinate Hermes versus Wolf/Moonlight.
  • Wolf ownership makes the gate return a deliberate unavailable response. The Hermes gateway stays alive and can use its configured fallback.
  • The two agent pods do not consume titan-24 GPU memory. Ollama does. Agent pods are ARM gateway workloads with persistent state on separate PVCs.
  • Fallback credentials are deliberately per-instance. Never copy Brad's Codex credential store into the consumer PVC.

Verify rather than memorize:

hermes status
hermes fallback list
kubectl -n hermes get configmap hermes-gpu-owner -o yaml
kubectl -n hermes get deploy hermes hermes-model-gate hermes-ollama -o wide
kubectl -n hermes-chat get deploy hermes-chat -o wide

Do not change the GPU owner as part of a lesson.

Operator evidence path

The operator role is designed for supervised test-failure triage:

  1. Ariadne deterministic bundle and optional diagnosis.
  2. Retained Jenkins logs and artifacts.
  3. Recent Git changes and Flux revision/state.
  4. Kubernetes workload, node, event, storage, DNS, and dependency health.
  5. Pushgateway quality metrics and VictoriaMetrics/Grafana context.
  6. A concise finding, confidence, evidence, blast radius, next checks, repo-side fix, and approval-required section.

The versioned $triage-titan-test-failures skill encodes this workflow. The operator has internal URLs for Ariadne, Jenkins, and VictoriaMetrics. The consumer instance intentionally does not.

Security layers and their limits

Security is layered rather than delegated to a prompt:

  1. Keycloak authenticates the human.
  2. The operator proxy limits agent.bstein.dev to Brad's exact email.
  3. Separate namespaces, PVCs, service accounts, configs, and ingresses isolate operator and consumer state.
  4. Kubernetes RBAC grants the consumer get/list/watch only and excludes Secrets, exec, attach, port-forward, and mutation.
  5. NetworkPolicy permits the consumer to reach DNS, the shared model gate, the Kubernetes API, and public IPv4 while blocking arbitrary private service access.
  6. Hermes deny patterns and instructions add user-facing guardrails.

Prompt rules and shell deny patterns are not the primary security boundary. RBAC, network policy, credential separation, and routing isolation are. The consumer can still disclose non-secret cluster metadata it is authorized to read; that is an intentional requirement and should be described honestly.

CLI-to-web mental model

The pinned Hermes build exposes chat, models and fallback, auth, sessions, profiles, skills and bundles, plugins, tools, MCP, gateway/channels, pairing, webhooks, cron, kanban, projects, hooks, memory/journey, logs, security, backup, and diagnostics. The web interface is a control surface over many of these same persisted capabilities.

Use <command> --help and the web Documentation page for the installed version. Do not rely on screenshots or online docs from a different release when a live command can settle the question.