Rename the owner agent host to worker.bstein.dev (supersedes #41) #42

Merged

bstein merged 3 commits from feature/hermes-domain-rename-agent-worker-v2 into main 2026-08-21 23:16:05 +00:00

Conversation 2 Commits 3 Files Changed 29 +2023 -121

hermes-automation commented 2026-08-21 10:44:32 +00:00

Member

Copy Link

Renames the owner-only Hermes coordinator host from agent.hermes.bstein.dev to
worker.bstein.dev.

Supersedes #41. That PR was cut before #38 landed and no longer merges: #38
touched the same five files. This branch is rebuilt on current main, and it now
reuses the legacy-origin mechanism #38 added instead of introducing a second one.

Read the rollout section before merging. This hostname is the coordinator's own
public access point, so the rollout — not the repo edit — changes how you reach the
running coordinator.

Deliberate design: the old host is not retired here

agent.hermes.bstein.dev keeps working through the transition. It stays in the
CoreDNS hosts entry, in BOTH agent Ingress rules, and on the hermes-sites-tls
certificate SANs. Retiring it is a separate, separately scheduled change.

No redirect middleware is added. A redirect from the old host to the new one could
be proposed later; it is deliberately not implemented here, so nothing silently
changes where a request lands.

This is the same additive shape #38 applied to chat and triage, applied to the agent
surface — and it is why this PR does not repeat #34's mistake.

The one thing that does move is where a fresh login happens. The oauth2-proxy
session and CSRF cookies use the __Host- prefix, which binds them to exactly one
origin, so --redirect-url can only name one host. It now names worker.bstein.dev.

Consequence, stated plainly:

• Already-authenticated sessions on agent.hermes.bstein.dev keep working.
• A new login started on the old host will bounce to the new host and fail CSRF
  validation, because the CSRF cookie was set on the old origin. Start fresh logins
  on worker.bstein.dev.
• Rollback is one line: revert --redirect-url in agent-deployment.yaml. It does
  NOT require re-running the Keycloak job, because both callback URLs stay
  registered.

Re-login required

oauth2-proxy cookies are host-bound (__Host-hermes_agent). Existing coordinator
sessions and cookies do NOT carry over to the new origin. Expect to log in again on
worker.bstein.dev after rollout, including any in-flight coordinator session.

Keycloak

services/keycloak/scripts/hermes_access_oidc_ensure.sh must run for the new
redirect URI. Login on worker.bstein.dev fails until it completes.

The agent client now passes a legacy origin through the optional fourth argument #38
added to ensure_proxy_client, so no new mechanism is introduced. The resulting
payload was replayed locally and registers both callbacks, both web origins, a
##-joined post-logout pair, and rootUrl on the canonical host.

The immutable ensure Job is bumped -11 -> -12. -11 was #38's bump and has
already run (verified live: chat and triage now accept their renamed callbacks),
so a further bump is required or this change would silently never be applied.

Ordered rollout

1. Merge this PR.
2. Flux reconciles infrastructure/core, services/hermes, services/keycloak,
   services/quality, services/comms.
3. CoreDNS serves worker.bstein.dev.
4. cert-manager reissues hermes-sites-tls; wait for Ready with the
   worker.bstein.dev SAN present.
5. The -12 ensure Job runs and updates the Keycloak agent proxy client.
6. Log in on https://worker.bstein.dev (re-login required, see above).
7. Only then consider retiring agent.hermes.bstein.dev, as a SEPARATE scheduled
   change. Do not retire it in this one.

Steps 3-5 are independent of each other but ALL must complete before step 6 succeeds.
If step 6 fails, the old host is still serving existing sessions and the one-line
rollback above applies.

Post-merge verification checklist

Each step marks whether this agent can run it. "Agent-executable" means the access
was demonstrated in this environment while preparing this PR, not assumed.

#	Check	Command	Pass signal	Who
1	New host resolves in cluster DNS	getent hosts worker.bstein.dev	prints 192.168.22.9 worker.bstein.dev	agent-executable (proven)
2	Old host still resolves	getent hosts agent.hermes.bstein.dev	prints 192.168.22.9 ...	agent-executable (proven)
3	New host routes through the ingress	curl -sS -o /dev/null -w '%{http_code}' https://worker.bstein.dev/ping	200	agent-executable (proven)
4	Old host still routes	curl -sS -o /dev/null -w '%{http_code}' https://agent.hermes.bstein.dev/ping	200	agent-executable (proven)
5	Certificate carries BOTH SANs	echo | openssl s_client -connect 192.168.22.9:443 -servername worker.bstein.dev 2>/dev/null | openssl x509 -noout -ext subjectAltName	lists worker.bstein.dev AND agent.hermes.bstein.dev	agent-executable (proven)
6	Keycloak accepts the new callback	authorization-endpoint probe for the agent proxy client with redirect uri https://worker.bstein.dev/oauth2/callback	HTTP 302, not 400 Invalid parameter: redirect_uri	agent-executable (proven)
7	Keycloak still accepts the legacy callback	same probe with the old callback URL	HTTP 302	agent-executable (proven)
8	Certificate resource is Ready	kubectl -n hermes get certificate hermes-sites-tls	READY=True	operator-required
9	Ingress objects show both hosts	kubectl -n hermes get ingress hermes-agent-dashboard hermes-agent-terminal	both hosts listed	operator-required
10	Ensure Job completed	kubectl -n sso get job hermes-access-oidc-client-ensure-12	COMPLETIONS=1/1	operator-required
11	Flux reconciled	flux get kustomizations	touched kustomizations Ready	operator-required
12	Browser login works end to end	log in at https://worker.bstein.dev	dashboard loads after re-login	operator-required (interactive)

Steps 8-11 are operator-required because this agent's Kubernetes identity
(system:serviceaccount:hermes:hermes-agent) is observer-only. Every one of
get ingress -n hermes, get certificate -n hermes, get job -n sso,
get configmap coredns-custom -n kube-system and get kustomization -n flux-system
returned Forbidden when tested. Do not expect the agent to confirm them.

Live state re-verified before this PR

Checks 1-7 were run against the cluster as it stands today, so the pre-merge baseline
is recorded rather than assumed:

• worker.bstein.dev -> NXDOMAIN; agent.hermes.bstein.dev -> resolves.
• The live hermes-sites-tls SAN list does not yet contain worker.bstein.dev.
• The agent proxy client accepts the legacy callback (302) and rejects
  https://worker.bstein.dev/oauth2/callback (400). Expected: this PR is what
  registers it.
• The chat and triage clients now accept their renamed callbacks (302). The
  breakage reported on #41 was real and #38 has since fixed it — no action needed
  here, and the "merge this last" advice is back to being purely about sequencing.

Scope and collisions

Does not touch chat.bstein.dev, triage.bstein.dev or their legacy names beyond
the shared certificate/Ingress files, where changes are strictly additive and #38's
entries are preserved verbatim.

Collision check before editing: worker.bstein.dev was unused — absent from
git grep over tracked content, from every knowledge catalog and Ingress, and
getent hosts returned NXDOMAIN.

Two commits, deliberately split:

1. chore(knowledge): regenerate stale Atlas catalogs — pure make knowledge run
   against unmodified main. The committed catalogs had drifted: #38's restored legacy
   hosts were missing from the HTTP catalogs and diagrams, along with Flux
   kustomizations and Grafana panels added since the last regeneration. Isolated so
   the rename reviews as a rename.
2. refactor(hermes): rename the owner agent host to worker.bstein.dev.

Intentionally left on the old hostname

• mermaid/ServiceAccessArchitecture.mmd and .svg. These two files cannot be
  pushed through the SCM broker at all: its content scanner has a forge-token rule
  matching a gitea-style prefix followed by a 20-plus character identifier suffix,
  and one pre-existing mermaid node id in these diagrams (the Cassandra OIDC access
  node) matches it, so the whole push is rejected with an opaque HTTP 400. That id is
  already on main, so any PR touching these two files hits the same wall regardless
  of what it changes. Diagnosed by replaying the broker's own scan over the outgoing
  objects locally. Fixing it is a scanner-tuning or diagram-regeneration concern, not
  a rename concern.
• testing/tests/data/hermes_kanban_card_corpus.jsonl. Its own test documents it as
  "a verbatim snapshot of every card on the board"; rewriting a historical record to
  match a later rename would falsify it.

Validation

• kustomize build clean for every touched service: services/hermes,
  services/keycloak, services/quality, services/comms, infrastructure/core.
• Rendered output spot-checked: both agent Ingresses serve both hosts, the
  certificate lists 6 SANs, the ensure Job renders as -12, and the agent
  --redirect-url and dashboard URL both point at the new host while chat and triage
  are untouched.
• sh -n clean on the ensure script; ruff clean on all touched Python.
• Keycloak payload replayed locally through #38's code path: both redirect URIs, both
  web origins, ##-joined post-logout, rootUrl canonical.
• API_SERVER_CORS_ORIGINS comma-splitting confirmed against the running Hermes
  build, so the paired value is valid.
• make test (full quality gate): 2641 passed, 4 failed, 4 skipped. All 4 failures
  reproduce identically on unmodified current main (5f9c600f) and are unrelated to
  this change: two node-capacity assertions in test_hermes_agent_layout.py, one
  test_hermes_gitea_pr_integration.py test that reaches the live network instead of
  its monkeypatch, and
  test_hermes_kanban_supervisor.py::test_auto_supervise_flag_defaults_false_in_configmap.
  No new failures.
• Test contract updated: the canonical-hostname tests assert the new host; a new test
  pins the agent transition contract (CoreDNS, both Ingresses and the certificate must
  carry BOTH hosts); and #38's continuity test now covers the agent proxy's dual
  origins instead of asserting the agent surface was untouched. Retiring the old host
  will deliberately fail these until the retirement change updates them.

Renames the owner-only Hermes coordinator host from `agent.hermes.bstein.dev` to `worker.bstein.dev`. **Supersedes #41.** That PR was cut before #38 landed and no longer merges: #38 touched the same five files. This branch is rebuilt on current main, and it now reuses the legacy-origin mechanism #38 added instead of introducing a second one. **Read the rollout section before merging.** This hostname is the coordinator's own public access point, so the rollout — not the repo edit — changes how you reach the running coordinator. ## Deliberate design: the old host is not retired here `agent.hermes.bstein.dev` keeps working through the transition. It stays in the CoreDNS hosts entry, in BOTH agent Ingress rules, and on the `hermes-sites-tls` certificate SANs. Retiring it is a separate, separately scheduled change. No redirect middleware is added. A redirect from the old host to the new one could be proposed later; it is deliberately not implemented here, so nothing silently changes where a request lands. This is the same additive shape #38 applied to chat and triage, applied to the agent surface — and it is why this PR does not repeat #34's mistake. The one thing that does move is where a *fresh login* happens. The oauth2-proxy session and CSRF cookies use the `__Host-` prefix, which binds them to exactly one origin, so `--redirect-url` can only name one host. It now names `worker.bstein.dev`. Consequence, stated plainly: - Already-authenticated sessions on `agent.hermes.bstein.dev` keep working. - A *new* login started on the old host will bounce to the new host and fail CSRF validation, because the CSRF cookie was set on the old origin. Start fresh logins on `worker.bstein.dev`. - Rollback is one line: revert `--redirect-url` in `agent-deployment.yaml`. It does NOT require re-running the Keycloak job, because both callback URLs stay registered. ## Re-login required oauth2-proxy cookies are host-bound (`__Host-hermes_agent`). Existing coordinator sessions and cookies do NOT carry over to the new origin. Expect to log in again on `worker.bstein.dev` after rollout, including any in-flight coordinator session. ## Keycloak `services/keycloak/scripts/hermes_access_oidc_ensure.sh` must run for the new redirect URI. **Login on `worker.bstein.dev` fails until it completes.** The agent client now passes a legacy origin through the optional fourth argument #38 added to `ensure_proxy_client`, so no new mechanism is introduced. The resulting payload was replayed locally and registers both callbacks, both web origins, a `##`-joined post-logout pair, and `rootUrl` on the canonical host. The immutable ensure Job is bumped `-11` -> `-12`. **-11 was #38's bump and has already run** (verified live: chat and triage now accept their renamed callbacks), so a further bump is required or this change would silently never be applied. ## Ordered rollout 1. Merge this PR. 2. Flux reconciles `infrastructure/core`, `services/hermes`, `services/keycloak`, `services/quality`, `services/comms`. 3. CoreDNS serves `worker.bstein.dev`. 4. cert-manager reissues `hermes-sites-tls`; wait for Ready with the `worker.bstein.dev` SAN present. 5. The `-12` ensure Job runs and updates the Keycloak agent proxy client. 6. Log in on `https://worker.bstein.dev` (re-login required, see above). 7. Only then consider retiring `agent.hermes.bstein.dev`, as a SEPARATE scheduled change. Do not retire it in this one. Steps 3-5 are independent of each other but ALL must complete before step 6 succeeds. If step 6 fails, the old host is still serving existing sessions and the one-line rollback above applies. ## Post-merge verification checklist Each step marks whether this agent can run it. "Agent-executable" means the access was demonstrated in this environment while preparing this PR, not assumed. | # | Check | Command | Pass signal | Who | |---|---|---|---|---| | 1 | New host resolves in cluster DNS | `getent hosts worker.bstein.dev` | prints `192.168.22.9 worker.bstein.dev` | agent-executable (proven) | | 2 | Old host still resolves | `getent hosts agent.hermes.bstein.dev` | prints `192.168.22.9 ...` | agent-executable (proven) | | 3 | New host routes through the ingress | `curl -sS -o /dev/null -w '%{http_code}' https://worker.bstein.dev/ping` | `200` | agent-executable (proven) | | 4 | Old host still routes | `curl -sS -o /dev/null -w '%{http_code}' https://agent.hermes.bstein.dev/ping` | `200` | agent-executable (proven) | | 5 | Certificate carries BOTH SANs | `echo \| openssl s_client -connect 192.168.22.9:443 -servername worker.bstein.dev 2>/dev/null \| openssl x509 -noout -ext subjectAltName` | lists `worker.bstein.dev` AND `agent.hermes.bstein.dev` | agent-executable (proven) | | 6 | Keycloak accepts the new callback | authorization-endpoint probe for the agent proxy client with redirect uri `https://worker.bstein.dev/oauth2/callback` | HTTP `302`, not `400 Invalid parameter: redirect_uri` | agent-executable (proven) | | 7 | Keycloak still accepts the legacy callback | same probe with the old callback URL | HTTP `302` | agent-executable (proven) | | 8 | Certificate resource is Ready | `kubectl -n hermes get certificate hermes-sites-tls` | `READY=True` | operator-required | | 9 | Ingress objects show both hosts | `kubectl -n hermes get ingress hermes-agent-dashboard hermes-agent-terminal` | both hosts listed | operator-required | | 10 | Ensure Job completed | `kubectl -n sso get job hermes-access-oidc-client-ensure-12` | `COMPLETIONS=1/1` | operator-required | | 11 | Flux reconciled | `flux get kustomizations` | touched kustomizations Ready | operator-required | | 12 | Browser login works end to end | log in at `https://worker.bstein.dev` | dashboard loads after re-login | operator-required (interactive) | Steps 8-11 are operator-required because this agent's Kubernetes identity (`system:serviceaccount:hermes:hermes-agent`) is observer-only. Every one of `get ingress -n hermes`, `get certificate -n hermes`, `get job -n sso`, `get configmap coredns-custom -n kube-system` and `get kustomization -n flux-system` returned `Forbidden` when tested. Do not expect the agent to confirm them. ## Live state re-verified before this PR Checks 1-7 were run against the cluster as it stands today, so the pre-merge baseline is recorded rather than assumed: - `worker.bstein.dev` -> NXDOMAIN; `agent.hermes.bstein.dev` -> resolves. - The live `hermes-sites-tls` SAN list does not yet contain `worker.bstein.dev`. - The agent proxy client accepts the legacy callback (`302`) and rejects `https://worker.bstein.dev/oauth2/callback` (`400`). Expected: this PR is what registers it. - The chat and triage clients now accept their renamed callbacks (`302`). The breakage reported on #41 was real and **#38 has since fixed it** — no action needed here, and the "merge this last" advice is back to being purely about sequencing. ## Scope and collisions Does not touch `chat.bstein.dev`, `triage.bstein.dev` or their legacy names beyond the shared certificate/Ingress files, where changes are strictly additive and #38's entries are preserved verbatim. Collision check before editing: `worker.bstein.dev` was unused — absent from `git grep` over tracked content, from every knowledge catalog and Ingress, and `getent hosts` returned NXDOMAIN. Two commits, deliberately split: 1. `chore(knowledge): regenerate stale Atlas catalogs` — pure `make knowledge` run against unmodified main. The committed catalogs had drifted: #38's restored legacy hosts were missing from the HTTP catalogs and diagrams, along with Flux kustomizations and Grafana panels added since the last regeneration. Isolated so the rename reviews as a rename. 2. `refactor(hermes): rename the owner agent host to worker.bstein.dev`. ## Intentionally left on the old hostname - `mermaid/ServiceAccessArchitecture.mmd` and `.svg`. These two files cannot be pushed through the SCM broker at all: its content scanner has a forge-token rule matching a `gitea`-style prefix followed by a 20-plus character identifier suffix, and one pre-existing mermaid node id in these diagrams (the Cassandra OIDC access node) matches it, so the whole push is rejected with an opaque HTTP 400. That id is already on main, so any PR touching these two files hits the same wall regardless of what it changes. Diagnosed by replaying the broker's own scan over the outgoing objects locally. Fixing it is a scanner-tuning or diagram-regeneration concern, not a rename concern. - `testing/tests/data/hermes_kanban_card_corpus.jsonl`. Its own test documents it as "a verbatim snapshot of every card on the board"; rewriting a historical record to match a later rename would falsify it. ## Validation - `kustomize build` clean for every touched service: `services/hermes`, `services/keycloak`, `services/quality`, `services/comms`, `infrastructure/core`. - Rendered output spot-checked: both agent Ingresses serve both hosts, the certificate lists 6 SANs, the ensure Job renders as `-12`, and the agent `--redirect-url` and dashboard URL both point at the new host while chat and triage are untouched. - `sh -n` clean on the ensure script; `ruff` clean on all touched Python. - Keycloak payload replayed locally through #38's code path: both redirect URIs, both web origins, `##`-joined post-logout, `rootUrl` canonical. - `API_SERVER_CORS_ORIGINS` comma-splitting confirmed against the running Hermes build, so the paired value is valid. - `make test` (full quality gate): **2641 passed, 4 failed, 4 skipped**. All 4 failures reproduce identically on unmodified current main (5f9c600f) and are unrelated to this change: two node-capacity assertions in `test_hermes_agent_layout.py`, one `test_hermes_gitea_pr_integration.py` test that reaches the live network instead of its monkeypatch, and `test_hermes_kanban_supervisor.py::test_auto_supervise_flag_defaults_false_in_configmap`. No new failures. - Test contract updated: the canonical-hostname tests assert the new host; a new test pins the agent transition contract (CoreDNS, both Ingresses and the certificate must carry BOTH hosts); and #38's continuity test now covers the agent proxy's dual origins instead of asserting the agent surface was untouched. Retiring the old host will deliberately fail these until the retirement change updates them.

hermes-automation added 2 commits 2026-08-21 10:44:33 +00:00

chore(knowledge): regenerate stale Atlas catalogs ... 004c41629e

`make knowledge` output on main no longer matched the manifests it renders
from. The legacy chat/triage hosts restored by #38 were missing from the
committed HTTP catalogs and diagrams, along with Flux kustomizations and
Grafana panels added since the last regeneration.

Pure `make knowledge` run against unmodified main, separated into its own
commit so the hostname rename that follows reviews as a hostname rename and
nothing else. No hand edits.

refactor(hermes): rename the owner agent host to worker.bstein.dev ... 94106bf252

Introduce worker.bstein.dev as the canonical hostname for the owner-only
Hermes coordinator, previously agent.hermes.bstein.dev.

The rename is additive, matching the shape #38 restored for chat and triage.
CoreDNS, both agent Ingresses and the hermes-sites certificate now serve BOTH
names, so merging this cannot take away the endpoint the operator uses to
reach the coordinator. Retiring agent.hermes.bstein.dev is a separate,
separately scheduled change. No redirect middleware is added.

What switches to the new host:
- HERMES_DASHBOARD_PUBLIC_URL and the oauth2-proxy --redirect-url
- the Keycloak agent proxy rootUrl
- operator docs, skills, the ZAP baseline target and the triage monitor default

What stays dual-homed until retirement:
- CoreDNS hosts entry, both agent Ingress rules, certificate SANs
- API_SERVER_CORS_ORIGINS (now a comma-separated pair)
- the Keycloak redirect URIs, web origins and post-logout origins, so a
  rollback only needs the oauth2-proxy --redirect-url reverted and does not
  require re-running the ensure job

The agent client passes its legacy origin through the optional fourth argument
#38 added to ensure_proxy_client, so no second mechanism is introduced. The
immutable ensure Job goes -11 -> -12 because #38 already consumed -11 and that
run has completed; without a further bump this change would never be applied.
Login on the new host fails until the -12 Job completes.

Because the session and CSRF cookies use the __Host- prefix they are bound to
one origin, so a fresh login must start on worker.bstein.dev and existing
sessions do not carry over -- re-login is required after rollout.

#38's public-host continuity test now covers the agent proxy's dual origins
rather than asserting the agent surface was untouched by the rename.

Knowledge catalogs and diagrams regenerated with `make knowledge`.

bstein approved these changes 2026-08-21 23:15:12 +00:00

Dismissed

bstein added 1 commit 2026-08-21 23:15:33 +00:00

Merge branch 'main' into feature/hermes-domain-rename-agent-worker-v2 ed98278981

bstein dismissed bstein’s review 2026-08-21 23:15:33 +00:00

Reason:

New commits pushed, approval review dismissed automatically according to repository settings

bstein changed title from WIP: Rename the owner agent host to worker.bstein.dev (supersedes #41) to Rename the owner agent host to worker.bstein.dev (supersedes #41) 2026-08-21 23:15:48 +00:00

bstein approved these changes 2026-08-21 23:16:00 +00:00

bstein merged commit 66c1c11d06 into main 2026-08-21 23:16:05 +00:00

bstein referenced this issue from a commit 2026-08-21 23:16:06 +00:00

Merge pull request 'Rename the owner agent host to worker.bstein.dev (supersedes #41)' (#42) from feature/hermes-domain-rename-agent-worker-v2 into main

Sign in to join this conversation.

Reviewers

No Reviewers

bstein

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

atlas-reconciler bstein hermes-automation

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: titan/atlas-iac#42

No description provided.