titan/atlas-iac
3
0
Fork 0
Code Issues Pull Requests 9 Actions Packages Projects Releases Wiki Activity

WIP: hermes: restore legacy chat/triage hosts alongside the renamed ones #37

Closed
hermes-automation wants to merge 0 commits from fix/hermes-restore-legacy-chat-triage-hosts into main
pull from: fix/hermes-restore-legacy-chat-triage-hosts
merge into: titan:main
Conversation 0 Commits - Files Changed -
hermes-automation commented 2026-08-21 08:50:13 +00:00
Member
Copy Link

Supersedes #36 (which only bumped the ensure Job and would have left the legacy hosts dark). Close #36 in favour of this one.

Why

Merged PR #34 renamed the chat/triage public hosts in place instead of adding the new names. It removed chat.hermes.bstein.dev and triage.hermes.bstein.dev from the certificate SANs, the hermes-sites Ingress rules and the CoreDNS overrides in a single change. Result, verified live on 2026-08-21:

  • Legacy hosts: HTTP 404 with Traefik's default self-signed certificate (no Ingress rule matches), and no longer resolvable in cluster CoreDNS.
  • Renamed hosts: TLS and routing are healthy (302 to Keycloak, oauth2-proxy /ping returns 200), but Keycloak answers the auth redirect with HTTP 400 "Invalid parameter" because hermes-chat-proxy and hermes-triage-proxy still hold the pre-rename callback URLs.

So chat and triage currently have no working hostname at all. The post-merge runbook's rollback stance for exactly this case is to reinstate the known-good old DNS/ingress/cert/OIDC targets rather than remove anything.

What this does

Makes the rename additive instead of destructive:

  • services/hermes/agent-certificate.yaml - legacy names back on hermes-sites-tls (5 SANs total; agent.hermes.bstein.dev preserved).
  • services/hermes/agent-ingress.yaml - legacy host rules restored, pointing at the same oauth2-proxy-hermes-chat and oauth2-proxy-hermes-triage backends, plus both names on the TLS block.
  • infrastructure/core/coredns-custom.yaml - both legacy host overrides restored for in-cluster resolution.
  • services/keycloak/scripts/hermes_access_oidc_ensure.sh - ensure_proxy_client takes an optional legacy origin and registers old and new callback URLs, web origins and post-logout origins together. rootUrl stays on the canonical new host. The single-argument call path (agent client) is byte-identical to before.
  • services/keycloak/bootstrap-jobs/hermes-access-oidc-client-job.yaml - Job bumped to ensure-11 so Flux recreates the immutable Job and reruns the script.

Serving both names is deliberate rather than redirecting: oauth2-proxy cookies are host-bound, so a redirect would silently drop live sessions. Retiring the legacy hosts stays a separate, explicit change.

Validation performed

  • sh -n on the ensure script; payload-generation logic extracted and run against jq 1.8.1. Dual-host output has both callbacks, both origins, rootUrl on the new host and hash-joined post-logout origins; single-host output matches current production semantics exactly.
  • kubectl kustomize services/hermes renders 5 cert SANs and all 4 host rules.
  • YAML parse checks on all four manifests; CoreDNS hosts block has no new duplicates.

Expected after merge

Flux reconciles, cert-manager reissues hermes-sites-tls covering all 5 names, and the ensure-11 Job registers both origins per client. Then all four hostnames serve 302 to Keycloak and reach the login form instead of the 400 error.

Note for operators: users must log in again on the renamed hosts because oauth2-proxy cookies are host-bound; sessions on the legacy hosts are preserved by this change.

Supersedes #36 (which only bumped the ensure Job and would have left the legacy hosts dark). Close #36 in favour of this one. ## Why Merged PR #34 renamed the chat/triage public hosts in place instead of adding the new names. It removed chat.hermes.bstein.dev and triage.hermes.bstein.dev from the certificate SANs, the hermes-sites Ingress rules and the CoreDNS overrides in a single change. Result, verified live on 2026-08-21: - Legacy hosts: HTTP 404 with Traefik's default self-signed certificate (no Ingress rule matches), and no longer resolvable in cluster CoreDNS. - Renamed hosts: TLS and routing are healthy (302 to Keycloak, oauth2-proxy /ping returns 200), but Keycloak answers the auth redirect with HTTP 400 "Invalid parameter" because hermes-chat-proxy and hermes-triage-proxy still hold the pre-rename callback URLs. So chat and triage currently have no working hostname at all. The post-merge runbook's rollback stance for exactly this case is to reinstate the known-good old DNS/ingress/cert/OIDC targets rather than remove anything. ## What this does Makes the rename additive instead of destructive: - services/hermes/agent-certificate.yaml - legacy names back on hermes-sites-tls (5 SANs total; agent.hermes.bstein.dev preserved). - services/hermes/agent-ingress.yaml - legacy host rules restored, pointing at the same oauth2-proxy-hermes-chat and oauth2-proxy-hermes-triage backends, plus both names on the TLS block. - infrastructure/core/coredns-custom.yaml - both legacy host overrides restored for in-cluster resolution. - services/keycloak/scripts/hermes_access_oidc_ensure.sh - ensure_proxy_client takes an optional legacy origin and registers old and new callback URLs, web origins and post-logout origins together. rootUrl stays on the canonical new host. The single-argument call path (agent client) is byte-identical to before. - services/keycloak/bootstrap-jobs/hermes-access-oidc-client-job.yaml - Job bumped to ensure-11 so Flux recreates the immutable Job and reruns the script. Serving both names is deliberate rather than redirecting: oauth2-proxy cookies are host-bound, so a redirect would silently drop live sessions. Retiring the legacy hosts stays a separate, explicit change. ## Validation performed - sh -n on the ensure script; payload-generation logic extracted and run against jq 1.8.1. Dual-host output has both callbacks, both origins, rootUrl on the new host and hash-joined post-logout origins; single-host output matches current production semantics exactly. - kubectl kustomize services/hermes renders 5 cert SANs and all 4 host rules. - YAML parse checks on all four manifests; CoreDNS hosts block has no new duplicates. ## Expected after merge Flux reconciles, cert-manager reissues hermes-sites-tls covering all 5 names, and the ensure-11 Job registers both origins per client. Then all four hostnames serve 302 to Keycloak and reach the login form instead of the 400 error. Note for operators: users must log in again on the renamed hosts because oauth2-proxy cookies are host-bound; sessions on the legacy hosts are preserved by this change.
hermes-automation added 1 commit 2026-08-21 08:50:14 +00:00 
PR #34 renamed the public chat/triage hosts in place rather than adding
the new names, so chat.hermes.bstein.dev and triage.hermes.bstein.dev
were dropped from the certificate SANs, the hermes-sites Ingress rules
and the CoreDNS overrides at once. Both legacy hosts now answer 404 with
Traefik's default self-signed certificate, and the renamed hosts cannot
complete a login because the Keycloak clients still carry the old
redirect URIs, so chat and triage are unreachable on every hostname.

Make the rename additive, which is the rollback path the post-merge
runbook asks for when the OIDC step fails:

- put the legacy names back on hermes-sites-tls and on the Ingress,
  pointing at the same oauth2-proxy backends
- restore both CoreDNS host overrides for in-cluster resolution
- teach ensure_proxy_client to register an optional legacy origin, so
  hermes-chat-proxy and hermes-triage-proxy accept the old and new
  redirect URIs, web origins and post-logout origins at the same time
  while rootUrl stays on the canonical new host
- bump the immutable ensure Job so Flux reruns the script

Serving both names is deliberate: oauth2-proxy cookies are host-bound,
so redirecting the legacy hosts would silently drop live sessions.
Retiring them stays a separate, explicit change.

Supersedes #36, which only bumped the Job and would have left the
legacy hosts dark.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
bstein closed this pull request 2026-08-22 20:22:27 +00:00

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
atlas-reconciler bstein hermes-automation
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: titan/atlas-iac#37
No description provided.
Delete Branch "fix/hermes-restore-legacy-chat-triage-hosts"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?