titan/atlas-iac
3
0
Fork 0
Code Issues Pull Requests 9 Actions Packages Projects Releases Wiki Activity

fix(hermes): poll the session contract the chat tenants actually serve #25

Merged
bstein merged 1 commits from fix/hermes-chat-session-continuity-contract into main 2026-08-20 21:55:17 +00:00
Conversation 1 Commits 1 Files Changed 6 +471 -81
hermes-automation commented 2026-08-20 18:16:57 +00:00
Member
Copy Link

Symptom

Returning to chat.hermes.bstein.dev after a Keycloak logout/login showed:

This session is unavailable to this account. Start a new chat.

…even though the session was intact and owned by the same Keycloak subject.

Root cause

The banner is rendered by the continuity fallback the chat router injects into
every page (services/hermes/router/session_continuity.go). It polled

  • GET /api/sessions/<id>/messages?limit=24&hermes_fallback=1
  • GET /api/sessions/<id>

Those routes belong to the Hermes agent dashboard — they are added by
services/hermes/scripts/patch_web_session_activity.py, which
agent-deployment.yaml applies and chat-statefulset.yaml does not.

The chat router proxies browser traffic to the tenant Hermes WebUI
(hermes-chat-tenant-N:8787) instead. Verified against the deployed image
registry.bstein.dev/bstein/hermes-webui@sha256:c276a9e1…: its only session
read is GET /api/session?session_id=<id>; /api/sessions/<id>[/messages] is
unrouted, so handle_get returns False and server.py answers its generic
404 {"error":"not found"} for every poll.

So the 404 branch fired unconditionally and reported an ownership failure that
had not occurred. It surfaced specifically on relogin because the script only
runs on a full document load of /session/<id> — exactly what the OIDC
round-trip produces when oauth2-proxy returns the browser to
rd=/session/<id>. Navigating within the SPA never re-runs it.

Slot assignment, the tenant header boundary and the cookie boundary were all
working correctly; nothing was actually mis-routed.

Fix

Poll the contract this backend serves, and let the WebUI's own answers decide
what the banner claims:

response meaning banner
200 session resolved hidden (or "Hermes is working…" while streaming)
409 session_profile_mismatch genuinely outside this account's active scope "belongs to a different profile"
404 no longer stored "no longer stored in your private chat"
401/403 SSO expired re-enter OIDC via /oauth2/start?rd= (unchanged)

boundSessionSnapshot follows the same move: it caps the WebUI envelope
{"session": {…, "messages": […]}} and relays every other session key
verbatim instead of re-serializing a fixed struct that would silently drop
metadata the banner depends on.

Steady state is now one request instead of two, backed off to 3s/15s — it
reaches a real endpoint on the tenant Raspberry Pi now, so the poll has a real
cost it did not have while it was 404ing.

Isolation

Unchanged, and now covered by tests. The router still resolves the slot from
the salted Keycloak subject, overwrites any client-supplied
X-Hermes-Tenant-Identity, and forwards only hermes_chat_session /
hermes_chat_profile.

Negative test: a second Keycloak subject replaying the owner's session id, the
owner's WebUI cookie and a forged tenant header is routed to its own slot,
gets 404, never sees the owner's payload, and never reaches the owner's
backend.

Tests

  • go test ./services/hermes/router — all pass; go vet and gofmt clean.
    New: TestLegacyDashboardSessionPollAlwaysMissesTheTenantWebUI,
    TestSessionContinuityPollSurvivesLogoutAndRelogin,
    TestSessionContinuityNeverExposesAnotherSubjectsSession,
    TestBoundSessionSnapshotRelaysUnknownSessionMetadata.
    The relogin/isolation tests derive the polled URL from the served asset, so
    reverting the script fails them rather than silently reintroducing the 404.
  • TDD evidence: the new tests fail against the pre-fix implementation
    ("continuity fallback does not build a session-scoped poll URL",
    session fallback omitted "/api/session?session_id=") and pass after.
  • pytest testing/tests/test_hermes_chat_session_continuity.py — 10 passed
    (2 new cross-file contract tests).
  • make test — 2620 passed, 4 skipped, 4 failed; all 4 failures reproduce
    unchanged on clean origin/main (test_hermes_agent_layout ×2,
    test_hermes_gitea_pr_integration, test_hermes_kanban_supervisor) and are
    unrelated to this diff.
  • kustomize build services/hermes — OK.
  • Script behaviour verified under Node with a stubbed DOM/fetch across all five
    response paths.

Deploy note

The router image is digest-pinned; this change ships when
hermes-chat-router is rebuilt. The injected script's cache-busting query is
bumped to v=20260820-webui-session-contract so browsers do not keep the
stale asset (Cache-Control: public, max-age=3600).

Domain rename is deliberately out of scope — no chat.hermes.bstein.dev
references were touched.

🤖 Generated with Claude Code

## Symptom Returning to `chat.hermes.bstein.dev` after a Keycloak logout/login showed: > This session is unavailable to this account. Start a new chat. …even though the session was intact and owned by the same Keycloak subject. ## Root cause The banner is rendered by the continuity fallback the chat router injects into every page (`services/hermes/router/session_continuity.go`). It polled - `GET /api/sessions/<id>/messages?limit=24&hermes_fallback=1` - `GET /api/sessions/<id>` Those routes belong to the **Hermes agent dashboard** — they are added by `services/hermes/scripts/patch_web_session_activity.py`, which `agent-deployment.yaml` applies and `chat-statefulset.yaml` does not. The chat router proxies browser traffic to the **tenant Hermes WebUI** (`hermes-chat-tenant-N:8787`) instead. Verified against the deployed image `registry.bstein.dev/bstein/hermes-webui@sha256:c276a9e1…`: its only session read is `GET /api/session?session_id=<id>`; `/api/sessions/<id>[/messages]` is unrouted, so `handle_get` returns `False` and `server.py` answers its generic `404 {"error":"not found"}` for **every** poll. So the 404 branch fired unconditionally and reported an ownership failure that had not occurred. It surfaced specifically on relogin because the script only runs on a **full document load** of `/session/<id>` — exactly what the OIDC round-trip produces when oauth2-proxy returns the browser to `rd=/session/<id>`. Navigating within the SPA never re-runs it. Slot assignment, the tenant header boundary and the cookie boundary were all working correctly; nothing was actually mis-routed. ## Fix Poll the contract this backend serves, and let the WebUI's own answers decide what the banner claims: | response | meaning | banner | |---|---|---| | `200` | session resolved | hidden (or "Hermes is working…" while streaming) | | `409 session_profile_mismatch` | genuinely outside this account's active scope | "belongs to a different profile" | | `404` | no longer stored | "no longer stored in your private chat" | | `401`/`403` | SSO expired | re-enter OIDC via `/oauth2/start?rd=` (unchanged) | `boundSessionSnapshot` follows the same move: it caps the WebUI envelope `{"session": {…, "messages": […]}}` and relays every other session key verbatim instead of re-serializing a fixed struct that would silently drop metadata the banner depends on. Steady state is now one request instead of two, backed off to 3s/15s — it reaches a real endpoint on the tenant Raspberry Pi now, so the poll has a real cost it did not have while it was 404ing. ## Isolation Unchanged, and now covered by tests. The router still resolves the slot from the salted Keycloak subject, overwrites any client-supplied `X-Hermes-Tenant-Identity`, and forwards only `hermes_chat_session` / `hermes_chat_profile`. Negative test: a second Keycloak subject replaying the owner's session id, the owner's WebUI cookie **and** a forged tenant header is routed to its own slot, gets `404`, never sees the owner's payload, and never reaches the owner's backend. ## Tests - `go test ./services/hermes/router` — all pass; `go vet` and `gofmt` clean. New: `TestLegacyDashboardSessionPollAlwaysMissesTheTenantWebUI`, `TestSessionContinuityPollSurvivesLogoutAndRelogin`, `TestSessionContinuityNeverExposesAnotherSubjectsSession`, `TestBoundSessionSnapshotRelaysUnknownSessionMetadata`. The relogin/isolation tests derive the polled URL from the served asset, so reverting the script fails them rather than silently reintroducing the 404. - TDD evidence: the new tests fail against the pre-fix implementation ("continuity fallback does not build a session-scoped poll URL", `session fallback omitted "/api/session?session_id="`) and pass after. - `pytest testing/tests/test_hermes_chat_session_continuity.py` — 10 passed (2 new cross-file contract tests). - `make test` — 2620 passed, 4 skipped, 4 failed; all 4 failures reproduce unchanged on clean `origin/main` (`test_hermes_agent_layout` ×2, `test_hermes_gitea_pr_integration`, `test_hermes_kanban_supervisor`) and are unrelated to this diff. - `kustomize build services/hermes` — OK. - Script behaviour verified under Node with a stubbed DOM/fetch across all five response paths. ## Deploy note The router image is digest-pinned; this change ships when `hermes-chat-router` is rebuilt. The injected script's cache-busting query is bumped to `v=20260820-webui-session-contract` so browsers do not keep the stale asset (`Cache-Control: public, max-age=3600`). Domain rename is deliberately out of scope — no `chat.hermes.bstein.dev` references were touched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
hermes-automation added 1 commit 2026-08-20 18:16:58 +00:00 
Returning to chat.hermes.bstein.dev after a Keycloak logout/login showed
"This session is unavailable to this account. Start a new chat." even
though the session was intact and owned by the same subject.

The banner comes from the continuity fallback the router injects into
every chat page. It polled `/api/sessions/<id>` and
`/api/sessions/<id>/messages` — routes that belong to the Hermes agent
dashboard (added by scripts/patch_web_session_activity.py, applied only
in agent-deployment.yaml). The router proxies browser traffic to the
tenant Hermes WebUI instead, whose only session read is
`GET /api/session?session_id=<id>`; the dashboard paths are unrouted
there, so server.py answered its generic 404 for every poll and the
fallback reported a false ownership failure.

The script runs only on a full document load of `/session/<id>`, which is
exactly what the OIDC round-trip produces when oauth2-proxy returns the
browser to `rd=/session/<id>` — hence the "only after relogin" symptom.

Poll the WebUI contract instead, and let its own answers decide what the
banner claims: 409 `session_profile_mismatch` is the single response that
means the session is outside this account's active scope, 404 now means
the conversation is no longer stored, and 401/403 still re-enter OIDC.
The steady-state poll drops to one request and backs off to 3s/15s now
that it reaches a real endpoint on the tenant Raspberry Pi.

`boundSessionSnapshot` follows the same move: it caps the WebUI envelope
`{"session": {..., "messages": [...]}}`, relaying every other session key
verbatim rather than re-serializing a fixed struct that would silently
drop metadata the banner depends on.

Isolation is unchanged and now covered: the router still resolves the
slot from the salted Keycloak subject, overwrites any client-supplied
X-Hermes-Tenant-Identity, and forwards only the two tenant cookies.

Tests: relogin keeps a stable slot and resolves the durable session; a
second subject replaying the owner's session id, WebUI cookie and a
forged tenant header gets 404 from its own backend and never reaches the
owner's; the legacy dashboard paths are pinned as permanent 404s against
a stub of the deployed WebUI dispatch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
bstein changed title from WIP: fix(hermes): poll the session contract the chat tenants actually serve to fix(hermes): poll the session contract the chat tenants actually serve 2026-08-20 21:45:36 +00:00
bstein approved these changes 2026-08-20 21:55:09 +00:00
bstein merged commit ac01fbd38a into main 2026-08-20 21:55:17 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
bstein
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
atlas-reconciler bstein hermes-automation
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: titan/atlas-iac#25
No description provided.
Delete Branch "fix/hermes-chat-session-continuity-contract"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?