Stdlib per-tenant service: trusted-header identity (router/relay/worker, constant-time keys, slot pinned to the pod), fail-closed card flags with capability negotiation, tenant-scoped store (atomic writes, revisions, append-only ledgers, content-addressed blobs, manifest), audit outcome for every request, and the /hux/v1 pipeline that maps errors to hux.error.v1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNPhwu2bsaRNg3DETSAZoM
24 KiB
hux-foundation threat model
Scope: the per-tenant hux-foundation service (Python, stdlib
ThreadingHTTPServer) that runs as a sidecar in each hermes-chat-tenant-N
pod, keeps HUX records on that pod's home PVC under /opt/data/hux/v1/
(layout in DATA-MODEL.md), and is reached only through the chat router,
the Worker, and the Telegram relay. It implements the frozen 1.0.0 contracts
in services/hermes/contracts/hux/ (ADR-0001) with the rules in
dockerfiles/hermes-hux-foundation/hux/rules.py. Where a NetworkPolicy or
pod setting is needed it is written as a requirement for Codex, not a
manifest. Every mitigation cites a contract field, a code obligation, or an
SO id from the checklist at the end.
Assets, most important first:
- Tenant isolation: one Keycloak subject, one pod, one PVC
(
chat-statefulset.yaml,ai.bstein.dev/isolation). Everything rests on it. - Secrets that transit the pod: the relay key at
/runtime-access/chat-relay-key, provider keys in/opt/data/.env, and the OAuth headers the router already strips (main.goDirector). - User content: memory, artifacts, sources, events. Sensitive and restricted
topics have hard rules in
PRIVACY_TOPICSandmemory_policy_violations. - Honesty of the record:
seq, approval decisions, cancel receipts, budget state, release evidence. If these can be forged the UI lies to the user.
Attackers, in the order we expect to meet them:
- A: another tenant's Hermes process or a tool it ran. Shell, Python, Playwright, and a same-namespace address.
- B: this tenant's own agent gone wrong (prompt injection via web or files). Tenant privileges, plus tool arguments and logs full of things the user never meant to keep.
- C: the tenant's browser session, possibly a stolen
hermes_chat_sessioncookie, driving/hux/v1through the router. - D: a linked Telegram peer, or a leaked relay key.
- E: operator mistakes: a flag on without its chain, an old binary reading new records, a purge with a bad glob.
1. Tenant and surface identity
The router is the only identity authority. slotFor maps subject to slot,
the Director strips Authorization, Cookie, X-Auth-Request-* and
X-Forwarded-*, and sets X-Hermes-Tenant-Identity: slot-N. Nothing in the
pod ever sees a raw subject. ADR-0001 adds X-Hux-Subject: usr_<hash>
(identityHash("keycloak", subject)) and X-Hux-Surface. The service
turns those into common.identity (tenant_slot, subject, surface,
trust) and stamps it on every record a caller can create.
| Threat | STRIDE | Who | Mitigation |
|---|---|---|---|
Pod-network client sends X-Hermes-Tenant-Identity: slot-2 straight to slot 2's hux port |
S | A | Bind 127.0.0.1 only; the WebUI sidecar is the sole in-pod caller and forwards the router's headers verbatim. No cross-pod path exists to spoof. (SO-01) |
| Someone reaches 8787/8642/8788 from a pod that is not the router | S | A | hermes-chat-tenant-isolation already admits those ports only from app: hermes-chat-router. Requirement: no hux port is ever added to that ingress list; the sandbox pods keep their single-source ingress. (SO-02) |
Browser sends X-Hux-Subject, router forwards it |
S | C | Router deletes inbound X-Hux-* and X-Hermes-Tenant-Identity before setting its own, exactly as it deletes Authorization. Service refuses a slot header that differs from its own ordinal (HERMES_TENANT_SLOT). (SO-03) |
Two header vocabularies (ADR: X-Hux-Subject; DATA-MODEL §6: X-Hermes-User) and one hop honours the wrong one |
S, E | E | One header set, defined once in identity.py, with the other names rejected as 401 unauthorized. (SO-04) |
| Telegram relay carries the shared relay key and no subject | S, E | D | Authorization: Bearer <relay_key> compared with hmac.compare_digest (as telegram_media_server.py does); X-Hux-Surface: telegram, trust: relay. The owner is the slot's single assignment. A relay request that names X-Hux-Subject is 401. (SO-05, SO-06) |
| Relay key ends up in a record or log | I | B | Key lives only under /runtime-access (read-only mount, not the PVC). It, and every value in /opt/data/.env, is a literal canary in the scrub of section 2. (SO-07) |
Worker reads a tenant's user data through trust: worker |
E, I | E | trust: worker may call GET /hux/v1/releases, GET /hux/v1/capabilities and POST /hux/v1/admin/retention only; any other route is 403 forbidden. No worker bypass port. (SO-08) |
| Slot handed to a new subject; old subtree leaks | I | E | Paths are users/<usr_hash>/...; reads join on identity.subject. A different hash sees an empty tree, never another's. (SO-09) |
Body claims provenance.surface: worker or actor.type: user |
R, T | B, C | Server overwrites identity, provenance.surface, provenance.actor (when type: user) and recorded_at from the trusted hop and its own clock. (SO-10) |
NetworkPolicy requirements for Codex: router-only ingress to every listening port in the tenant pod; no egress from hux-foundation (it fetches nothing); sandbox pods gain no route to the tenant's loopback services.
2. Activity events
detail is schema-open ("kind-specific payload"). Tool arguments carry file
paths, shell lines with tokens, URLs with query strings, pasted secrets.
The pipeline, in order, before any byte is written: drop keys outside the
per-kind allowlist; scrub every remaining string; cap sizes; raise
redaction.level if anything fired; then and only then allocate seq.
| Threat | STRIDE | Who | Mitigation |
|---|---|---|---|
tool.call detail persists raw arguments |
I | B | Allowlist below; unknown keys dropped before write, never filtered on read only. (SO-11) |
Secret inside an allowed string (summary, detail.target_path) |
I | B | Pattern scrub: sk-[A-Za-z0-9]{20,}, `ghp_ |
| Oversized detail fills the PVC or an SSE buffer | D | B | detail > 32 KiB (DATA-MODEL cap) is replaced by {"truncated": true, "bytes": N}; line cap 64 KiB; evidence ≤ 64 and summary ≤ 280 per schema. (SO-13) |
Reader ignores redaction.level |
I | C, D | Enforced at serve time too: none as stored; partial strips detail; full serves kind, seq, ts, sensitivity, redaction only. surface: telegram and voice never receive none. (SO-14) |
Client supplies seq or id |
T | B, C | Server-assigned under the conversation lock; a body carrying either is 400 invalid. (SO-15) |
Replay through Idempotency-Key returns someone else's event |
I | C | Idempotency keys are scoped per conversation file; a replay is served only after the ownership check. (SO-16) |
Reconnect storm, after_seq=0 on a long log |
D | C | Pages ≤ 200; after_seq before the retention window resumes from the oldest kept event; one SSE stream per (subject, conversation), a second open closes the first; idle timeout 15 min. (SO-17) |
GET /conversations/{guess}/events |
I | C | Ids are time-prefixed plus 30 random bits, so ownership is the real gate: conversation_id document loaded first, mismatch is 404 not_found, not 403. (SO-18) |
evidence[].uri of file:///opt/data/.env expanded by a UI |
I | B | Service never dereferences evidence_ref.uri; stored URIs must be https://, hux:// or artifact://, others dropped. (SO-19) |
| Restricted event leaks via search | I | C | sensitivity: restricted or redaction.level: full events never enter search/<conv>.json. (SO-20) |
Detail allowlist (anything else is dropped):
message.*:message_id,chars,has_attachmentsdecision.route:requested,resolved_target,provider,effort,reasondecision.plan:steps(≤ 20 strings ≤ 200 chars)tool.call:tool,capability,argument_names,argument_hash,target_path(kept only under/opt/data/workspace, else dropped)tool.result:tool,ok,duration_ms,bytes,exit_codeapproval.*,side_effect.*:approval_id,capability,choice,externalmemory.*:memory_id,kind,sensitivity,topicartifact.*:artifact_id,version,type,bytes,hashrun.*,delegation.*,budget.exhausted:run_id,outcome,receipt_id,spent,limits,child_run_idprivacy.notice: thehux.privacy_notice.v1fields onlycitation.attached,mode.changed,suggestion.*,release.transition: ids and enum fields of their own schemas only
3. Memory
Defaults: approval_mode: ask for anything above personal; no_store is
a first-class status (memory.status, memory.approval_mode) so "do not
remember" is recorded as a decision rather than as absence. A proposal
unanswered for 7 days expires.
| Threat | STRIDE | Who | Mitigation |
|---|---|---|---|
| Restricted content written because the topic detector missed it | I | B | memory_policy_violations on every write and transition; any problem is 422/invalid with a memory.suppressed event. The service never lowers sensitivity; its own scrub may raise it. (SO-21) |
| "Do not remember" blocks the write but the entry surfaces later | I | B, C | no_store writes a content-free ledger line and a tombstone; forget writes status: forgotten, content: "", retrievable: false, tombstone. Retrieval, /search, /export and the agent read hook check the tombstone set before the index and return only status: active with retrievable: true. (SO-22, SO-23) |
| Forgotten content lingers in earlier events or ledger lines | I | B | memory.* events carry ids only. forget re-redacts events referencing the id to full and queues purge_forgotten_content to rewrite older ledger snapshots. (SO-24) |
| Correction resurrects forgotten text | T | B | An entry whose supersedes or source points at a forgotten id needs approval_mode: ask answered by a user actor. (SO-25) |
| Export leaks another owner or forgotten entries | I | C | Export is owner-scoped, excludes forgotten, rejected, expired, no_store and retrievable: false, appends exported to audit[], is served as an attachment, and the snapshot file is pruned after 7 days. (SO-26) |
| TTL never enforced because the audit job died | I | E | Expiry checked lazily at read (an expired entry is never served as active) and eagerly by the retention job; a missing hux.retention_audit.v1 older than 48 h makes /privacy/policy report audit_stale: true. (SO-27) |
| Private mode writes memory | I | B | MODE_CATALOG["private"].memory.write == False: POST /memory is refused while the conversation's mode is private, and its events are not written at all (204). (SO-28) |
disable_memory_here ignored on the read side |
I | B | privacy/conversation_topics.json memory_disabled is checked by retrieval: that conversation contributes and receives nothing. (SO-23) |
4. Artifacts and research
Sources are records about where evidence came from. The service stores and serves them; it never fetches a URI.
| Threat | STRIDE | Who | Mitigation |
|---|---|---|---|
SSRF via source.uri or evidence_ref.uri |
S, I | B | No outbound HTTP client in the service; a test asserts urllib.request, http.client and socket.create_connection are absent from the module graph. (SO-29) |
| Wrong hash on upload; UI dedupes on it | T | B | sha256 recomputed server-side; mismatch with content_ref.hash is 422; blobs stored at blobs/sha256/<aa>/<hash>, mode 0400, never rewritten. (SO-30) |
| Oversized artifact | D | B, C | 25 MiB per version, 200 versions, 2 000 artifacts, 2 GiB blob store per tenant (DATA-MODEL caps); over limit is 413 too_large or 409 conflict. (SO-31) |
| HTML/SVG served executable into the chat origin | I, E | B | Blobs served as application/octet-stream, Content-Disposition: attachment, X-Content-Type-Options: nosniff. mime is metadata for the UI's sandboxed preview, never a response header. (SO-32) |
| Lineage or promotion names a record the caller does not own | T, I | B, C | lineage.artifact_id, promotion.project_id, passage.source_id, citation.passage_ids, notebook id lists all resolved under the caller's subtree at write time; unknown or foreign is 422. There is no cross-tenant store, and the check stays explicit so a future shared cache cannot widen it. (SO-33) |
access.mode: shared_readonly used as a cross-tenant channel |
I | C | Share tokens are minted by the owner, read-only, expire at share_expires_at ≤ 7 days, and resolve only inside the same pod; the router does not route them to another slot. (SO-34) |
Passage text used to smuggle secrets into the citation strip |
I | B | Passage text passes the SO-12 scrub; 4 000-char cap is the schema's. (SO-12) |
5. Autonomy
The service is the policy store and approval queue; the gateway executes
tools. The gate is only as strong as the gateway's call to
effective_decision and its presentation of the approval id at execution.
| Threat | STRIDE | Who | Mitigation |
|---|---|---|---|
| Agent decides its own approval | S, E | B | POST /approvals/{id} needs identity.surface in {chat, telegram, voice} and trust in {router, relay}; the stored decision.by is the asserted user, not the body's actor. (SO-35) |
once reused for two side effects |
T | B | Approvals are terminal after one transition (APPROVAL_TRANSITIONS); consume marks _meta.consumed_at and a second consume of once is 409 conflict. (SO-36) |
| TOCTOU: approved "write notes.md", executed "write ~/.ssh/authorized_keys" | T | B | request.evidence carries a tool_call ref whose hash is the sha256 of the canonical argument JSON; consume must present the same hash or it is denied with a side_effect.blocked event. (SO-37) |
always grant never expires |
E | C | always writes a grant with server-set expires_at ≤ 30 days and granted_by; the client cannot extend it. (SO-38) |
deploy, external_side_effect or network slip through under safe |
E | B, C | effective_decision is the only resolver; _ALWAYS_ASK ends ask, unexpired deny wins. request.external: true always requires an approval record. (SO-39) |
| Budget exhausted by delegation fan-out | D | B | hux.budget_state.v1 is updated per tool.result/delegation.*; once exhausted the service refuses new approvals with 402-style budget_exhausted and emits budget.exhausted. (SO-40) |
Receipt says cancelled while a shell still runs |
R | B | outcome: cancelled only after the gateway reports the run's process registry empty; else failed_to_cancel. side_effects must include every tool.result with ok: true after requested_at, each marked reverted. Stop is not done until the receipt exists. (SO-41) |
| Approval sits open forever | D | B | expires_at ≤ 24 h from requested_at; expiry is a terminal transition written by the retention thread. (SO-42) |
6. Storage
Layout per DATA-MODEL §1: /opt/data/hux/v1/users/<usr_hash>/<family>/....
No table, index or cache is shared between tenants because no two tenants
share a PVC.
| Threat | STRIDE | Who | Mitigation |
|---|---|---|---|
| Path traversal via id | I, T | B, C | store.path_for validates the user hash and every id against the common.schema.json patterns (/ impossible), then os.path.commonpath against the user subtree, as normalizeTenantMediaPath does. No caller string reaches a path any other way. (SO-43) |
| Lost update between chat and Telegram | T | C, D | revision served as ETag; PATCH/PUT need If-Match, mismatch 409 conflict; writes are temp + os.replace + fsync under the family lock. Unconditional writes are audited why: unconditional_write. (SO-44) |
| Agent edits the audit log; it has write on the PVC | R, T | B | hux-foundation runs as its own uid (not 10000); /opt/data/hux is 0700 to that uid; the agent reaches records only through the API. Requirement for Codex: separate runAsUser, readOnlyRootFilesystem, drop: [ALL]. (SO-45) |
| Audit line removed or altered | R | B, E | audit/outcomes/*.jsonl lines carry prev = sha256 of the previous line; a break is reported in the next retention audit as report. Audit lines never contain record content. (SO-46) |
| Purge deletes the wrong thing | D | E | purge_forgotten_content touches only tombstoned ids and blobs absent from refs.json for 7 days, writes the dry-run count into the retention record before the delete pass, and never touches audit/. (SO-47) |
| Second replica interleaves appends | T | E | v1/.lock taken with flock(LOCK_EX|LOCK_NB) at start-up; failure is fatal. (SO-48) |
| PVC full stops the WebUI too | D | B | Per-family bounds from DATA-MODEL §4; a write over cap is 413 too_large, and total /opt/data/hux above 4 GiB puts /healthz at degraded. (SO-49) |
7. Feature flags
| Threat | STRIDE | Who | Mitigation |
|---|---|---|---|
hux.memory_control on, hux.privacy off |
E | E | flag_enabled walks depends_on; a route serves only when the chain is on; /hux/v1 is 404 flag_off without hux.foundation. GET /capabilities reports the resolved chain, not the raw env. (SO-50) |
| Old binary reads new records | D | E | Additive-only within 1.x; _meta stripped on serve; unknown schema skipped on list, 409 on fetch; MANIFEST.json min_reader_layout refuses a tree the binary cannot read. (SO-51) |
| Rollback leaves data the disabled UI cannot show | I | E | Disabling hides routes, not data; TTL, forget and purge keep running. (SO-52) |
8. Abuse and DoS bounds
Sized for 4-8 tenants, one human each, on a 10 Gi PVC shared with the WebUI.
- Bodies: 64 KiB JSON, 25 MiB blob, rejected before parsing.
- Rate: 30 writes/min and 300 reads/min per subject,
429 rate_limitedwithRetry-After. Relay and worker get the same limits per slot. - Events: 50 000 per conversation then archived; 1 GiB total.
- Memory: 5 000 live entries; proposals expire in 7 days; ledger compacts at 64 MiB.
- Artifacts: 2 000, 200 versions each, 2 GiB blobs.
- Research: 10 000 sources, 50 000 passages, 512 MiB.
- Approvals: 50 pending per run; expiry ≤ 24 h.
- SSE: one per (subject, conversation); idle timeout 15 min; 8 per subject.
- Threads:
ThreadingHTTPServercapped at 32 concurrent requests; beyond that503.
Security obligations checklist
- SO-01 The service binds 127.0.0.1 only; any other bind address fails at startup.
- SO-02 Tenant pod ingress admits
app: hermes-chat-routeronly; no hux port is added to any NetworkPolicy. - SO-03 The router deletes inbound
X-Hux-*andX-Hermes-Tenant-Identitybefore setting its own; the service rejects a slot header not equal toHERMES_TENANT_SLOT. - SO-04 Exactly one identity header set is accepted (
X-Hermes-Tenant-Identity,X-Hux-Subject,X-Hux-Surface); any other vocabulary is401. - SO-05 Relay requests are checked with
hmac.compare_digestagainst the relay key and carrytrust: relay. - SO-06 A relay request carrying
X-Hux-Subjectis401; the owner is derived from the slot. - SO-07 The relay key and every
/opt/data/.envvalue never appear in a record, event detail, audit line or log. - SO-08
trust: workermay call only/releases,/capabilitiesand/admin/retention; everything else is403. - SO-09 Every path contains the asserted
usr_hash; a different subject on the same pod sees an empty tree. - SO-10
identity,provenance.surface, user-typeprovenance.actorandrecorded_atare server-set and client values ignored. - SO-11 Event
detailkeys outside the per-kind allowlist are dropped before persistence. - SO-12 Every stored string passes the secret scrub; a hit becomes
[redacted:<class>]and raisesredaction.levelto at leastpartial. - SO-13
detailover 32 KiB is replaced by a truncation marker; a line over 64 KiB is rejected. - SO-14
redaction.levelis enforced at serve time; telegram and voice never receivenone. - SO-15
seqandidare server-assigned; a body containing either is400. - SO-16 Idempotent replays are served only after the ownership check passes.
- SO-17 Event pages are ≤ 200; one SSE stream per (subject, conversation); idle timeout 15 min.
- SO-18 A conversation the caller does not own returns
404, never403. - SO-19 The service never dereferences
evidence_ref.uriorsource.uri; stored URIs use an allowlisted scheme. - SO-20 Restricted or fully redacted events never enter the search index.
- SO-21
memory_policy_violationsruns on every memory write and transition; a violation is422plus amemory.suppressedevent. - SO-22
no_storeandforgetwrite a content-free ledger line and a tombstone before returning. - SO-23 Retrieval, search, export and the agent read hook consult tombstones and
memory_disabledfirst and return onlyactiveentries withretrievable: true. - SO-24
forgetre-redacts events referencing the memory id tofulland queues the ledger rewrite. - SO-25 An entry that
supersedesor is sourced from a forgotten id requires a user-actor approval. - SO-26 Export is owner-scoped, excludes non-active entries, appends
exportedtoaudit[], and its snapshot is pruned after 7 days. - SO-27 Expired entries are never served as active; a retention audit older than 48 h is reported as
audit_stale. - SO-28 In
privatemodePOST /memoryis refused and event appends return204without writing. - SO-29 A test asserts no outbound HTTP or raw socket client is importable from the service's module graph.
- SO-30 Blob hashes are recomputed on upload; a mismatch is
422; blobs are 0400 and never rewritten. - SO-31 Artifact caps: 25 MiB per version, 200 versions, 2 000 artifacts, 2 GiB blobs.
- SO-32 Blobs are served as
application/octet-streamattachments withnosniff. - SO-33 Every referenced id (lineage, promotion, source, passage, citation, notebook) must resolve under the caller's subtree.
- SO-34 Share tokens are owner-minted, read-only, expire within 7 days, and resolve only on the owning pod.
- SO-35 Approvals are decided only from a human surface with
trustrouter or relay;decision.byis the asserted user. - SO-36 A
onceapproval can be consumed exactly once. - SO-37 Consume must present the argument hash recorded in
request.evidenceat request time. - SO-38
alwaysgrants carry a server-setexpires_atof at most 30 days. - SO-39
effective_decisionis the sole resolver;deployandexternal_side_effectalways resolve toask;request.external: truealways needs an approval record. - SO-40 An exhausted
budget_stateblocks new approvals withbudget_exhaustedand emitsbudget.exhausted. - SO-41 A receipt says
cancelledonly after the run's process registry is empty, and lists every successful side effect afterrequested_at. - SO-42 Pending approvals expire within 24 h of
requested_at. - SO-43 All paths go through
path_for: pattern-validated ids pluscommonpathcontainment in the user subtree. - SO-44 Revisioned writes require
If-Match; mismatch is409; unconditional writes are audited. - SO-45 The service runs as a distinct uid with
/opt/data/huxmode 0700; the agent has no filesystem path to records. - SO-46 Audit outcome lines are hash-chained and a broken chain is reported in the next retention audit.
- SO-47 Purge touches only tombstoned ids and unreferenced blobs older than 7 days, records a dry-run count first, and never touches
audit/. - SO-48 A second service process against the same tree fails at startup on
v1/.lock. - SO-49 Writes beyond family caps are
413; total store above 4 GiB marks/healthzdegraded. - SO-50 A route serves only when its flag and every dependency flag are on;
/capabilitiesreports the resolved chain. - SO-51 Responses strip
_meta, unknownschemavalues are skipped on list and409on fetch, and a tree abovemin_reader_layoutis refused. - SO-52 Disabling a flag hides routes; TTL, forget and purge continue.
- SO-53 Rate limits (30 writes/min, 300 reads/min per subject) return
429withRetry-After. - SO-54 Bodies over 64 KiB (JSON) or 25 MiB (blob) are rejected with
413before parsing.