2026-08-17 13:11:34 +00:00
|
|
|
# Hermes full-handoff acceptance and release runbook
|
hermes: add a fail-closed full-handoff acceptance harness
Decides whether the Hermes platform handoff is fit to release, and refuses
to round an absence of evidence up to a pass.
The harness is read-only by default and classifies 71 checks PASS / FAIL /
NOT_RUN / NOT_APPLICABLE. Any mandatory FAIL or NOT_RUN is NO_GO, and so is a
harness-level problem: an unreachable vantage, a catalog entry whose evidence
no longer exists, an expired deadline, or an evaluator that raised.
Evidence comes from two vantages that cannot cover for each other: an external
read-only operator kubeconfig, and the Hermes agent probing itself from inside
its own pod. Before any check runs, the harness asks each vantage who it is and
stops if they are the same principal, because dual-vantage evidence from one
identity is a restatement rather than a corroboration. `--as` is rejected for
every operator-side command and reachable only as the inner command of a
`kubectl exec`, so impersonation can never stand in for a real self-probe. A
deny check needs a live refused request, not only an authorization review.
Two safety properties are structural rather than conventional, enforced where
an argv becomes a subprocess: the default mode mutates nothing (mutating verbs
require a server dry run; there is deliberately no live TokenRequest probe,
because a successful one would mint a real credential), and no probe can pull a
credential value into a report (no vault/sops/curl, secrets readable only with
-o name, environment probes list names, shell only through frozen reviewed
templates). Captures are bounded before they are screened, and the rendered
report is re-screened before it is written.
Mutation lives behind a separate arming flag with an exact confirmation phrase,
a caller-supplied unique ref, a preflight that refuses a protected push target
before any network call, and a cleanup whose verification is itself mandatory.
A default run reports those four checks NOT_RUN.
The catalog is declarative so a reviewer reads what is asserted rather than how
it is plumbed, and so structural properties can be proven over every entry
before a run. Catalog drift surfaces as NOT_RUN, which stops the release.
docs/hermes_full_handoff_acceptance.md carries the merge order for PRs #14-#18
on top of the merged #13 baseline, the image build and Flux rollout, the
rollback point for each step, the go/no-go checklist, and the limits that are
asserted rather than exercised.
Validation: 295 handoff tests pass with 100% line coverage on all 15 new
modules; the full unit suite is 647 passed with two failures that reproduce
unchanged on origin/main; Ruff, py_compile, kustomize render, and a diff
credential screen are clean; a live read-only run against Atlas returns NO_GO
for the pre-merge cluster with no unscreened fields in the report.
2026-08-17 10:14:17 +00:00
|
|
|
|
2026-08-17 13:11:34 +00:00
|
|
|
`scripts/ops/hermes_handoff_acceptance.py` is a fail-closed release gate. It
|
|
|
|
|
contains 63 checks in 15 groups. An empty catalog, an unknown result status, a
|
|
|
|
|
mandatory `FAIL` or `NOT_RUN`, a harness error, or incomplete evidence makes the
|
|
|
|
|
decision `NO_GO`. The four ephemeral checks are non-mandatory `NOT_RUN` in the
|
|
|
|
|
default mode and are replaced with mandatory results only in armed mode.
|
hermes: add a fail-closed full-handoff acceptance harness
Decides whether the Hermes platform handoff is fit to release, and refuses
to round an absence of evidence up to a pass.
The harness is read-only by default and classifies 71 checks PASS / FAIL /
NOT_RUN / NOT_APPLICABLE. Any mandatory FAIL or NOT_RUN is NO_GO, and so is a
harness-level problem: an unreachable vantage, a catalog entry whose evidence
no longer exists, an expired deadline, or an evaluator that raised.
Evidence comes from two vantages that cannot cover for each other: an external
read-only operator kubeconfig, and the Hermes agent probing itself from inside
its own pod. Before any check runs, the harness asks each vantage who it is and
stops if they are the same principal, because dual-vantage evidence from one
identity is a restatement rather than a corroboration. `--as` is rejected for
every operator-side command and reachable only as the inner command of a
`kubectl exec`, so impersonation can never stand in for a real self-probe. A
deny check needs a live refused request, not only an authorization review.
Two safety properties are structural rather than conventional, enforced where
an argv becomes a subprocess: the default mode mutates nothing (mutating verbs
require a server dry run; there is deliberately no live TokenRequest probe,
because a successful one would mint a real credential), and no probe can pull a
credential value into a report (no vault/sops/curl, secrets readable only with
-o name, environment probes list names, shell only through frozen reviewed
templates). Captures are bounded before they are screened, and the rendered
report is re-screened before it is written.
Mutation lives behind a separate arming flag with an exact confirmation phrase,
a caller-supplied unique ref, a preflight that refuses a protected push target
before any network call, and a cleanup whose verification is itself mandatory.
A default run reports those four checks NOT_RUN.
The catalog is declarative so a reviewer reads what is asserted rather than how
it is plumbed, and so structural properties can be proven over every entry
before a run. Catalog drift surfaces as NOT_RUN, which stops the release.
docs/hermes_full_handoff_acceptance.md carries the merge order for PRs #14-#18
on top of the merged #13 baseline, the image build and Flux rollout, the
rollback point for each step, the go/no-go checklist, and the limits that are
asserted rather than exercised.
Validation: 295 handoff tests pass with 100% line coverage on all 15 new
modules; the full unit suite is 647 passed with two failures that reproduce
unchanged on origin/main; Ruff, py_compile, kustomize render, and a diff
credential screen are clean; a live read-only run against Atlas returns NO_GO
for the pre-merge cluster with no unscreened fields in the report.
2026-08-17 10:14:17 +00:00
|
|
|
|
2026-08-17 13:11:34 +00:00
|
|
|
This runbook describes evidence collection. It does not authorize a merge,
|
|
|
|
|
publish, deployment, Flux reconcile, credential read, or live cluster mutation.
|
|
|
|
|
Human review of PR #19 and of the resulting report is required.
|
|
|
|
|
|
|
|
|
|
## Required release inputs
|
|
|
|
|
|
|
|
|
|
Every value that can go stale must be supplied explicitly. Resolve `main`, PR
|
|
|
|
|
heads, the image digest, build source SHA, deployment revision, and chat config
|
|
|
|
|
revision from their authoritative read-only sources immediately before the run.
|
|
|
|
|
Do not copy the snapshot in the dependency section into a later run.
|
|
|
|
|
|
|
|
|
|
```sh
|
hermes: add a fail-closed full-handoff acceptance harness
Decides whether the Hermes platform handoff is fit to release, and refuses
to round an absence of evidence up to a pass.
The harness is read-only by default and classifies 71 checks PASS / FAIL /
NOT_RUN / NOT_APPLICABLE. Any mandatory FAIL or NOT_RUN is NO_GO, and so is a
harness-level problem: an unreachable vantage, a catalog entry whose evidence
no longer exists, an expired deadline, or an evaluator that raised.
Evidence comes from two vantages that cannot cover for each other: an external
read-only operator kubeconfig, and the Hermes agent probing itself from inside
its own pod. Before any check runs, the harness asks each vantage who it is and
stops if they are the same principal, because dual-vantage evidence from one
identity is a restatement rather than a corroboration. `--as` is rejected for
every operator-side command and reachable only as the inner command of a
`kubectl exec`, so impersonation can never stand in for a real self-probe. A
deny check needs a live refused request, not only an authorization review.
Two safety properties are structural rather than conventional, enforced where
an argv becomes a subprocess: the default mode mutates nothing (mutating verbs
require a server dry run; there is deliberately no live TokenRequest probe,
because a successful one would mint a real credential), and no probe can pull a
credential value into a report (no vault/sops/curl, secrets readable only with
-o name, environment probes list names, shell only through frozen reviewed
templates). Captures are bounded before they are screened, and the rendered
report is re-screened before it is written.
Mutation lives behind a separate arming flag with an exact confirmation phrase,
a caller-supplied unique ref, a preflight that refuses a protected push target
before any network call, and a cleanup whose verification is itself mandatory.
A default run reports those four checks NOT_RUN.
The catalog is declarative so a reviewer reads what is asserted rather than how
it is plumbed, and so structural properties can be proven over every entry
before a run. Catalog drift surfaces as NOT_RUN, which stops the release.
docs/hermes_full_handoff_acceptance.md carries the merge order for PRs #14-#18
on top of the merged #13 baseline, the image build and Flux rollout, the
rollback point for each step, the go/no-go checklist, and the limits that are
asserted rather than exercised.
Validation: 295 handoff tests pass with 100% line coverage on all 15 new
modules; the full unit suite is 647 passed with two failures that reproduce
unchanged on origin/main; Ruff, py_compile, kustomize render, and a diff
credential screen are clean; a live read-only run against Atlas returns NO_GO
for the pre-merge cluster with no unscreened fields in the report.
2026-08-17 10:14:17 +00:00
|
|
|
scripts/ops/hermes_handoff_acceptance.py \
|
|
|
|
|
--context atlas-operator \
|
2026-08-17 13:11:34 +00:00
|
|
|
--remote-main-sha "$REMOTE_MAIN_SHA" \
|
|
|
|
|
--reviewed-head-sha "$REVIEWED_PR19_HEAD" \
|
|
|
|
|
--agent-image "$AGENT_GIT_SHA_BUILD_TAG_AT_SHA256" \
|
|
|
|
|
--build-sha "$IMAGE_BUILD_SOURCE_SHA" \
|
|
|
|
|
--deployment-revision "$RUNNING_DEPLOYMENT_REVISION" \
|
|
|
|
|
--chat-config-revision "$RUNNING_CHAT_CONFIG_REVISION" \
|
|
|
|
|
--dependency-head "12=$PR12_HEAD" \
|
|
|
|
|
--dependency-head "14=$PR14_HEAD" \
|
|
|
|
|
--dependency-head "15=$PR15_HEAD" \
|
|
|
|
|
--dependency-head "16=$PR16_HEAD" \
|
|
|
|
|
--dependency-head "17=$PR17_HEAD" \
|
|
|
|
|
--dependency-head "18=$PR18_HEAD" \
|
|
|
|
|
--dependency-head "20=$PR20_HEAD" \
|
hermes: add a fail-closed full-handoff acceptance harness
Decides whether the Hermes platform handoff is fit to release, and refuses
to round an absence of evidence up to a pass.
The harness is read-only by default and classifies 71 checks PASS / FAIL /
NOT_RUN / NOT_APPLICABLE. Any mandatory FAIL or NOT_RUN is NO_GO, and so is a
harness-level problem: an unreachable vantage, a catalog entry whose evidence
no longer exists, an expired deadline, or an evaluator that raised.
Evidence comes from two vantages that cannot cover for each other: an external
read-only operator kubeconfig, and the Hermes agent probing itself from inside
its own pod. Before any check runs, the harness asks each vantage who it is and
stops if they are the same principal, because dual-vantage evidence from one
identity is a restatement rather than a corroboration. `--as` is rejected for
every operator-side command and reachable only as the inner command of a
`kubectl exec`, so impersonation can never stand in for a real self-probe. A
deny check needs a live refused request, not only an authorization review.
Two safety properties are structural rather than conventional, enforced where
an argv becomes a subprocess: the default mode mutates nothing (mutating verbs
require a server dry run; there is deliberately no live TokenRequest probe,
because a successful one would mint a real credential), and no probe can pull a
credential value into a report (no vault/sops/curl, secrets readable only with
-o name, environment probes list names, shell only through frozen reviewed
templates). Captures are bounded before they are screened, and the rendered
report is re-screened before it is written.
Mutation lives behind a separate arming flag with an exact confirmation phrase,
a caller-supplied unique ref, a preflight that refuses a protected push target
before any network call, and a cleanup whose verification is itself mandatory.
A default run reports those four checks NOT_RUN.
The catalog is declarative so a reviewer reads what is asserted rather than how
it is plumbed, and so structural properties can be proven over every entry
before a run. Catalog drift surfaces as NOT_RUN, which stops the release.
docs/hermes_full_handoff_acceptance.md carries the merge order for PRs #14-#18
on top of the merged #13 baseline, the image build and Flux rollout, the
rollback point for each step, the go/no-go checklist, and the limits that are
asserted rather than exercised.
Validation: 295 handoff tests pass with 100% line coverage on all 15 new
modules; the full unit suite is 647 passed with two failures that reproduce
unchanged on origin/main; Ruff, py_compile, kustomize render, and a diff
credential screen are clean; a live read-only run against Atlas returns NO_GO
for the pre-merge cluster with no unscreened fields in the report.
2026-08-17 10:14:17 +00:00
|
|
|
--output build/hermes-handoff-acceptance.json
|
|
|
|
|
```
|
|
|
|
|
|
2026-08-17 13:11:34 +00:00
|
|
|
The fixed topology is three pool workers on three distinct nodes. `--node-count`
|
|
|
|
|
and `--pool-replicas` cannot lower that requirement. Pool node/ordinal assignment,
|
|
|
|
|
Telegram session continuity, and the chat revision are mandatory. Deliberate Flux
|
|
|
|
|
suspensions must be named with repeatable `--expected-suspension` arguments.
|
|
|
|
|
|
|
|
|
|
Exit status is `0` only for `GO`, `1` for `NO_GO`, and `2` for invalid execution
|
|
|
|
|
bounds. JSON is written to `--output` or stdout; the human summary goes to stderr
|
|
|
|
|
unless `--json-only` is passed.
|
|
|
|
|
|
|
|
|
|
## Structural safety boundary
|
|
|
|
|
|
|
|
|
|
The catalog is validated before a `Runner` exists. Invalid inputs, an empty or
|
|
|
|
|
unsafe catalog, and an invalid armed preflight therefore create no subprocess.
|
|
|
|
|
The command policy is checked both before and after executable resolution.
|
|
|
|
|
|
|
|
|
|
Default mode permits a narrow read grammar:
|
|
|
|
|
|
|
|
|
|
- Kubernetes reads use `name` or reviewed JSONPath projections. Secret resources,
|
|
|
|
|
raw API reads, full JSON/YAML, `describe`, `cluster-info dump`, credential-bearing
|
|
|
|
|
projections, `auth reconcile`, config operations, all mutations, and every
|
|
|
|
|
dry-run mutation trick are rejected before spawn.
|
|
|
|
|
- Git is read-only. `fetch`, configuration/helper overrides, alternate worktrees,
|
fix(hermes): close the zero-evidence fail-open in absence checks
evaluate_names_absent returned PASS when its step exited 0 with no output,
so five mandatory checks - the ones asserting that provider API keys, forge
credentials, a cluster-admin binding, and shared coordinator state are
absent - could report a pass on no evidence and turn a NO_GO into a GO.
Both name rules now resolve their step through one guard in _line_step, so
zero observations are NOT_RUN. Regressions pin all five real catalog specs
plus both reachable silence paths: a POSIX pipeline whose status comes from
its last stage, and a drifted kubectl -o jsonpath. The pool claim projection
emits one <volume>=<claim> line per template volume so a volume without a
PVC still counts as an observation rather than reading as drift.
Also closes the review's reachable hardening and evidence defects:
- pin Gitea paths to atlas/titan-iac on an exact segment boundary and
reject relative segments, including percent-encoded ones
- forbid impersonation structurally in every mode and vantage; the inner
command of kubectl exec is re-checked rather than exempted, and
validate_catalog no longer guards only the operator vantage
- drop flux and helm from the binary allowlist; they had no pinned release
digest, so no allowlisted binary can now be admitted that the executor
would refuse to attest
- remove the inert --concurrency and --expect-telegram-sessions flags and
the dead concurrency bound; Telegram continuity stays mandatory
- read the ephemeral pull index page by page, treat the create response as
an authoritative source for the pull number, close every number either
source names, and surface residue_ref plus exact manual_cleanup commands
when creation is uncertain
- keep executable_path and executable_sha256 on unrecorded bulk-evidence
steps so withholding bytes never withholds binary attestation
- revert the repo-wide hygiene legacy-exception mechanism; the contract
change here is purely additive and the four pre-existing over-cap files
are left to the canonical contract change in PR #14/#15
- correct the runbook ruff format scope so the documented command passes
Split hermes_handoff_arming.py out of hermes_handoff_ephemeral.py to keep
both modules under the 500-line cap. All 16 handoff modules hold at least
95% line and branch coverage; the mutation gate is 13/13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 16:24:37 +00:00
|
|
|
external diff helpers, and push are rejected.
|
|
|
|
|
- Gitea paths are pinned to `atlas/titan-iac` on an exact segment boundary, so a
|
|
|
|
|
look-alike repository such as `atlas/titan-iac-evil` is refused, and any
|
|
|
|
|
relative segment — including a percent-encoded `%2e%2e` — is rejected before
|
|
|
|
|
the request is built.
|
2026-08-17 13:11:34 +00:00
|
|
|
- Shell commands must be rendered from frozen templates, use fixed executable
|
|
|
|
|
paths, and cannot name service-account, Vault, runtime-access, SSH, or other
|
|
|
|
|
credential roots.
|
fix(hermes): close the zero-evidence fail-open in absence checks
evaluate_names_absent returned PASS when its step exited 0 with no output,
so five mandatory checks - the ones asserting that provider API keys, forge
credentials, a cluster-admin binding, and shared coordinator state are
absent - could report a pass on no evidence and turn a NO_GO into a GO.
Both name rules now resolve their step through one guard in _line_step, so
zero observations are NOT_RUN. Regressions pin all five real catalog specs
plus both reachable silence paths: a POSIX pipeline whose status comes from
its last stage, and a drifted kubectl -o jsonpath. The pool claim projection
emits one <volume>=<claim> line per template volume so a volume without a
PVC still counts as an observation rather than reading as drift.
Also closes the review's reachable hardening and evidence defects:
- pin Gitea paths to atlas/titan-iac on an exact segment boundary and
reject relative segments, including percent-encoded ones
- forbid impersonation structurally in every mode and vantage; the inner
command of kubectl exec is re-checked rather than exempted, and
validate_catalog no longer guards only the operator vantage
- drop flux and helm from the binary allowlist; they had no pinned release
digest, so no allowlisted binary can now be admitted that the executor
would refuse to attest
- remove the inert --concurrency and --expect-telegram-sessions flags and
the dead concurrency bound; Telegram continuity stays mandatory
- read the ephemeral pull index page by page, treat the create response as
an authoritative source for the pull number, close every number either
source names, and surface residue_ref plus exact manual_cleanup commands
when creation is uncertain
- keep executable_path and executable_sha256 on unrecorded bulk-evidence
steps so withholding bytes never withholds binary attestation
- revert the repo-wide hygiene legacy-exception mechanism; the contract
change here is purely additive and the four pre-existing over-cap files
are left to the canonical contract change in PR #14/#15
- correct the runbook ruff format scope so the documented command passes
Split hermes_handoff_arming.py out of hermes_handoff_ephemeral.py to keep
both modules under the 500-line cap. All 16 handoff modules hold at least
95% line and branch coverage; the mutation gate is 13/13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 16:24:37 +00:00
|
|
|
- Impersonation is structurally impossible. `--as`, `--as-group`, and `--as-uid`
|
|
|
|
|
are refused for every argv, in every mode, from every vantage, and the inner
|
|
|
|
|
command of a `kubectl exec` is re-checked under the same rule rather than being
|
|
|
|
|
granted an exemption. `validate_catalog` refuses an impersonating step from any
|
|
|
|
|
vantage, not only the operator. A future audited self-probe that genuinely needs
|
|
|
|
|
impersonation is a reviewed code change, not a configuration flag.
|
|
|
|
|
- The binary allowlist is exactly the set with pinned release digests:
|
|
|
|
|
`kubectl`, `git`, `sh`, `hermes`, and the Gitea client. `flux` and `helm` are
|
|
|
|
|
not on it; Flux and Helm status is read through `kubectl` instead, so no
|
|
|
|
|
allowlisted binary can be admitted that the executor would refuse to attest.
|
2026-08-17 13:11:34 +00:00
|
|
|
- `PATH` is replaced with a fixed path. Every outer executable is resolved to an
|
|
|
|
|
allowed real path, checked for unsafe write mode, and compared with a pinned
|
|
|
|
|
SHA-256 digest. A caller-authored `kubectl`, `git`, or helper cannot fabricate
|
|
|
|
|
evidence.
|
|
|
|
|
|
|
|
|
|
The executor uses a new process group, drains stdout and stderr concurrently,
|
|
|
|
|
and retains at most the configured bytes while the child is running. A monotonic
|
|
|
|
|
absolute timeout terminates and reaps the group, including descendants that keep
|
fix(hermes): close the zero-evidence fail-open in absence checks
evaluate_names_absent returned PASS when its step exited 0 with no output,
so five mandatory checks - the ones asserting that provider API keys, forge
credentials, a cluster-admin binding, and shared coordinator state are
absent - could report a pass on no evidence and turn a NO_GO into a GO.
Both name rules now resolve their step through one guard in _line_step, so
zero observations are NOT_RUN. Regressions pin all five real catalog specs
plus both reachable silence paths: a POSIX pipeline whose status comes from
its last stage, and a drifted kubectl -o jsonpath. The pool claim projection
emits one <volume>=<claim> line per template volume so a volume without a
PVC still counts as an observation rather than reading as drift.
Also closes the review's reachable hardening and evidence defects:
- pin Gitea paths to atlas/titan-iac on an exact segment boundary and
reject relative segments, including percent-encoded ones
- forbid impersonation structurally in every mode and vantage; the inner
command of kubectl exec is re-checked rather than exempted, and
validate_catalog no longer guards only the operator vantage
- drop flux and helm from the binary allowlist; they had no pinned release
digest, so no allowlisted binary can now be admitted that the executor
would refuse to attest
- remove the inert --concurrency and --expect-telegram-sessions flags and
the dead concurrency bound; Telegram continuity stays mandatory
- read the ephemeral pull index page by page, treat the create response as
an authoritative source for the pull number, close every number either
source names, and surface residue_ref plus exact manual_cleanup commands
when creation is uncertain
- keep executable_path and executable_sha256 on unrecorded bulk-evidence
steps so withholding bytes never withholds binary attestation
- revert the repo-wide hygiene legacy-exception mechanism; the contract
change here is purely additive and the four pre-existing over-cap files
are left to the canonical contract change in PR #14/#15
- correct the runbook ruff format scope so the documented command passes
Split hermes_handoff_arming.py out of hermes_handoff_ephemeral.py to keep
both modules under the 500-line cap. All 16 handoff modules hold at least
95% line and branch coverage; the mutation gate is 13/13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 16:24:37 +00:00
|
|
|
pipes open. Timeout, deadline, and output-byte values reject booleans,
|
|
|
|
|
zero/negative values, non-finite numbers, and values outside fixed bounds. There
|
|
|
|
|
is no concurrency knob: `run_catalog` is strictly sequential and says so rather
|
|
|
|
|
than accepting a bound it would ignore.
|
2026-08-17 13:11:34 +00:00
|
|
|
Truncation discards the affected stream and makes every evaluator `NOT_RUN`.
|
fix(hermes): close the zero-evidence fail-open in absence checks
evaluate_names_absent returned PASS when its step exited 0 with no output,
so five mandatory checks - the ones asserting that provider API keys, forge
credentials, a cluster-admin binding, and shared coordinator state are
absent - could report a pass on no evidence and turn a NO_GO into a GO.
Both name rules now resolve their step through one guard in _line_step, so
zero observations are NOT_RUN. Regressions pin all five real catalog specs
plus both reachable silence paths: a POSIX pipeline whose status comes from
its last stage, and a drifted kubectl -o jsonpath. The pool claim projection
emits one <volume>=<claim> line per template volume so a volume without a
PVC still counts as an observation rather than reading as drift.
Also closes the review's reachable hardening and evidence defects:
- pin Gitea paths to atlas/titan-iac on an exact segment boundary and
reject relative segments, including percent-encoded ones
- forbid impersonation structurally in every mode and vantage; the inner
command of kubectl exec is re-checked rather than exempted, and
validate_catalog no longer guards only the operator vantage
- drop flux and helm from the binary allowlist; they had no pinned release
digest, so no allowlisted binary can now be admitted that the executor
would refuse to attest
- remove the inert --concurrency and --expect-telegram-sessions flags and
the dead concurrency bound; Telegram continuity stays mandatory
- read the ephemeral pull index page by page, treat the create response as
an authoritative source for the pull number, close every number either
source names, and surface residue_ref plus exact manual_cleanup commands
when creation is uncertain
- keep executable_path and executable_sha256 on unrecorded bulk-evidence
steps so withholding bytes never withholds binary attestation
- revert the repo-wide hygiene legacy-exception mechanism; the contract
change here is purely additive and the four pre-existing over-cap files
are left to the canonical contract change in PR #14/#15
- correct the runbook ruff format scope so the documented command passes
Split hermes_handoff_arming.py out of hermes_handoff_ephemeral.py to keep
both modules under the 500-line cap. All 16 handoff modules hold at least
95% line and branch coverage; the mutation gate is 13/13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 16:24:37 +00:00
|
|
|
Bulk-evidence steps have their bytes replaced before they are recorded, but they
|
|
|
|
|
keep the resolved executable path and its attested SHA-256, so withholding output
|
|
|
|
|
never withholds the provenance of the binary that produced it.
|
2026-08-17 13:11:34 +00:00
|
|
|
|
|
|
|
|
Credential screening runs before truncation. Command, output, error, identity,
|
|
|
|
|
reason, evidence, JSON, and human-summary fields are screened recursively,
|
|
|
|
|
including mapping keys and short/numeric sensitive values. The complete rendered
|
|
|
|
|
report is screened again; a remaining credential shape is replaced by a bounded
|
|
|
|
|
`NO_GO` stub rather than being written. No credential prefix is retained at a
|
|
|
|
|
truncation boundary.
|
|
|
|
|
|
|
|
|
|
## Vantages and evidence
|
|
|
|
|
|
|
|
|
|
The operator must use an external read-only kubeconfig. The self probe runs in
|
|
|
|
|
the actual Hermes agent pod. Both identities are obtained with
|
|
|
|
|
`kubectl auth whoami`; equal or missing principals make the run `NO_GO`. Required
|
|
|
|
|
Switchyard and chat/Telegram pod vantages must also resolve. Node coverage is
|
|
|
|
|
derived from all daemonset pod node names, not one selected pod.
|
|
|
|
|
|
|
|
|
|
Dangerous Kubernetes permissions are checked only with side-effect-free
|
|
|
|
|
`kubectl auth can-i` reviews. The default catalog never constructs a Secret,
|
|
|
|
|
TokenRequest, mutation, attach, port-forward, or arbitrary exec request. The one
|
|
|
|
|
forge administrative denial check performs a read-only GET and accepts only an
|
|
|
|
|
explicit authorization refusal; generic `404`, `not found`, and `no route` text
|
|
|
|
|
are not denial evidence.
|
|
|
|
|
|
fix(hermes): close the zero-evidence fail-open in absence checks
evaluate_names_absent returned PASS when its step exited 0 with no output,
so five mandatory checks - the ones asserting that provider API keys, forge
credentials, a cluster-admin binding, and shared coordinator state are
absent - could report a pass on no evidence and turn a NO_GO into a GO.
Both name rules now resolve their step through one guard in _line_step, so
zero observations are NOT_RUN. Regressions pin all five real catalog specs
plus both reachable silence paths: a POSIX pipeline whose status comes from
its last stage, and a drifted kubectl -o jsonpath. The pool claim projection
emits one <volume>=<claim> line per template volume so a volume without a
PVC still counts as an observation rather than reading as drift.
Also closes the review's reachable hardening and evidence defects:
- pin Gitea paths to atlas/titan-iac on an exact segment boundary and
reject relative segments, including percent-encoded ones
- forbid impersonation structurally in every mode and vantage; the inner
command of kubectl exec is re-checked rather than exempted, and
validate_catalog no longer guards only the operator vantage
- drop flux and helm from the binary allowlist; they had no pinned release
digest, so no allowlisted binary can now be admitted that the executor
would refuse to attest
- remove the inert --concurrency and --expect-telegram-sessions flags and
the dead concurrency bound; Telegram continuity stays mandatory
- read the ephemeral pull index page by page, treat the create response as
an authoritative source for the pull number, close every number either
source names, and surface residue_ref plus exact manual_cleanup commands
when creation is uncertain
- keep executable_path and executable_sha256 on unrecorded bulk-evidence
steps so withholding bytes never withholds binary attestation
- revert the repo-wide hygiene legacy-exception mechanism; the contract
change here is purely additive and the four pre-existing over-cap files
are left to the canonical contract change in PR #14/#15
- correct the runbook ruff format scope so the documented command passes
Split hermes_handoff_arming.py out of hermes_handoff_ephemeral.py to keep
both modules under the 500-line cap. All 16 handoff modules hold at least
95% line and branch coverage; the mutation gate is 13/13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 16:24:37 +00:00
|
|
|
An absence is only evidence when the probe observed something. Both name rules
|
|
|
|
|
resolve their step through one guard, so a step that exits `0` with no lines — or
|
|
|
|
|
with only separators — is `NOT_RUN`, never `PASS`. This is reachable in practice
|
|
|
|
|
two ways, and both are pinned by regressions: a POSIX pipeline reports the status
|
|
|
|
|
of its *last* stage, so a missing upstream binary in the frozen `env_names`
|
|
|
|
|
template exits `0` with no bytes; and `kubectl -o jsonpath` exits `0` when the
|
|
|
|
|
structure it walks has drifted, whether a parent field, a leaf field, or a
|
|
|
|
|
container filter. Because all five absence checks are mandatory, silence costs
|
|
|
|
|
the release its `GO`. The pool claim projection emits one `<volume>=<claim>` line
|
|
|
|
|
per template volume so a volume without a PVC still counts as an observation.
|
|
|
|
|
|
2026-08-17 13:11:34 +00:00
|
|
|
The strict evaluators reject duplicate-key JSON, raw control characters,
|
|
|
|
|
`NaN`/infinity, boolean/number equality coercion, future timestamps, stale
|
|
|
|
|
provider evidence, and every malformed routing or Flux row. Routing freshness is
|
|
|
|
|
required separately for each provider, lane, and effort. Flux objects must be
|
|
|
|
|
non-suspended, current at `observedGeneration`, and explicitly `Ready=True`;
|
|
|
|
|
unknown or reconciling state is not healthy.
|
|
|
|
|
|
|
|
|
|
`release.exact-lineage-is-running` atomically binds:
|
|
|
|
|
|
|
|
|
|
- fixed remote `main` SHA and local `origin/main`;
|
|
|
|
|
- open draft PR #19, base `main`, existing feature ref, and exact reviewed head;
|
|
|
|
|
- the exact `:git-<source-SHA>-build-<n>@sha256:<digest>` image identity,
|
|
|
|
|
with the source SHA equal to the release `main` SHA;
|
|
|
|
|
- current Deployment generation and revision;
|
|
|
|
|
- the one active ReplicaSet revision/hash; every Ready pod image and imageID;
|
|
|
|
|
- the Flux-applied `main` revision.
|
|
|
|
|
|
|
|
|
|
The pool evaluator requires exactly ordinals 0, 1, and 2, all Ready, on three
|
|
|
|
|
distinct nodes and claims, with service-account token automount disabled. The
|
|
|
|
|
chat checks always require durable Telegram sessions and the exact running chat
|
|
|
|
|
configuration revision.
|
|
|
|
|
|
|
|
|
|
## Armed ephemeral mode
|
|
|
|
|
|
|
|
|
|
Armed mode is the only write path and runs only after the complete default
|
|
|
|
|
catalog reports `GO`.
|
|
|
|
|
|
|
|
|
|
```sh
|
hermes: add a fail-closed full-handoff acceptance harness
Decides whether the Hermes platform handoff is fit to release, and refuses
to round an absence of evidence up to a pass.
The harness is read-only by default and classifies 71 checks PASS / FAIL /
NOT_RUN / NOT_APPLICABLE. Any mandatory FAIL or NOT_RUN is NO_GO, and so is a
harness-level problem: an unreachable vantage, a catalog entry whose evidence
no longer exists, an expired deadline, or an evaluator that raised.
Evidence comes from two vantages that cannot cover for each other: an external
read-only operator kubeconfig, and the Hermes agent probing itself from inside
its own pod. Before any check runs, the harness asks each vantage who it is and
stops if they are the same principal, because dual-vantage evidence from one
identity is a restatement rather than a corroboration. `--as` is rejected for
every operator-side command and reachable only as the inner command of a
`kubectl exec`, so impersonation can never stand in for a real self-probe. A
deny check needs a live refused request, not only an authorization review.
Two safety properties are structural rather than conventional, enforced where
an argv becomes a subprocess: the default mode mutates nothing (mutating verbs
require a server dry run; there is deliberately no live TokenRequest probe,
because a successful one would mint a real credential), and no probe can pull a
credential value into a report (no vault/sops/curl, secrets readable only with
-o name, environment probes list names, shell only through frozen reviewed
templates). Captures are bounded before they are screened, and the rendered
report is re-screened before it is written.
Mutation lives behind a separate arming flag with an exact confirmation phrase,
a caller-supplied unique ref, a preflight that refuses a protected push target
before any network call, and a cleanup whose verification is itself mandatory.
A default run reports those four checks NOT_RUN.
The catalog is declarative so a reviewer reads what is asserted rather than how
it is plumbed, and so structural properties can be proven over every entry
before a run. Catalog drift surfaces as NOT_RUN, which stops the release.
docs/hermes_full_handoff_acceptance.md carries the merge order for PRs #14-#18
on top of the merged #13 baseline, the image build and Flux rollout, the
rollback point for each step, the go/no-go checklist, and the limits that are
asserted rather than exercised.
Validation: 295 handoff tests pass with 100% line coverage on all 15 new
modules; the full unit suite is 647 passed with two failures that reproduce
unchanged on origin/main; Ruff, py_compile, kustomize render, and a diff
credential screen are clean; a live read-only run against Atlas returns NO_GO
for the pre-merge cluster with no unscreened fields in the report.
2026-08-17 10:14:17 +00:00
|
|
|
scripts/ops/hermes_handoff_acceptance.py \
|
2026-08-17 13:11:34 +00:00
|
|
|
<all required release inputs above> \
|
hermes: add a fail-closed full-handoff acceptance harness
Decides whether the Hermes platform handoff is fit to release, and refuses
to round an absence of evidence up to a pass.
The harness is read-only by default and classifies 71 checks PASS / FAIL /
NOT_RUN / NOT_APPLICABLE. Any mandatory FAIL or NOT_RUN is NO_GO, and so is a
harness-level problem: an unreachable vantage, a catalog entry whose evidence
no longer exists, an expired deadline, or an evaluator that raised.
Evidence comes from two vantages that cannot cover for each other: an external
read-only operator kubeconfig, and the Hermes agent probing itself from inside
its own pod. Before any check runs, the harness asks each vantage who it is and
stops if they are the same principal, because dual-vantage evidence from one
identity is a restatement rather than a corroboration. `--as` is rejected for
every operator-side command and reachable only as the inner command of a
`kubectl exec`, so impersonation can never stand in for a real self-probe. A
deny check needs a live refused request, not only an authorization review.
Two safety properties are structural rather than conventional, enforced where
an argv becomes a subprocess: the default mode mutates nothing (mutating verbs
require a server dry run; there is deliberately no live TokenRequest probe,
because a successful one would mint a real credential), and no probe can pull a
credential value into a report (no vault/sops/curl, secrets readable only with
-o name, environment probes list names, shell only through frozen reviewed
templates). Captures are bounded before they are screened, and the rendered
report is re-screened before it is written.
Mutation lives behind a separate arming flag with an exact confirmation phrase,
a caller-supplied unique ref, a preflight that refuses a protected push target
before any network call, and a cleanup whose verification is itself mandatory.
A default run reports those four checks NOT_RUN.
The catalog is declarative so a reviewer reads what is asserted rather than how
it is plumbed, and so structural properties can be proven over every entry
before a run. Catalog drift surfaces as NOT_RUN, which stops the release.
docs/hermes_full_handoff_acceptance.md carries the merge order for PRs #14-#18
on top of the merged #13 baseline, the image build and Flux rollout, the
rollback point for each step, the go/no-go checklist, and the limits that are
asserted rather than exercised.
Validation: 295 handoff tests pass with 100% line coverage on all 15 new
modules; the full unit suite is 647 passed with two failures that reproduce
unchanged on origin/main; Ruff, py_compile, kustomize render, and a diff
credential screen are clean; a live read-only run against Atlas returns NO_GO
for the pre-merge cluster with no unscreened fields in the report.
2026-08-17 10:14:17 +00:00
|
|
|
--arm-ephemeral-push \
|
|
|
|
|
--confirm "ARM EPHEMERAL HERMES HANDOFF PUSH" \
|
|
|
|
|
--ephemeral-token acceptance-20260817a \
|
|
|
|
|
--output build/hermes-handoff-acceptance-armed.json
|
|
|
|
|
```
|
|
|
|
|
|
2026-08-17 13:11:34 +00:00
|
|
|
Before any runner, network request, or cluster vantage is created, preflight
|
|
|
|
|
binds the working directory to the existing linked PR #19 worktree, the exact
|
|
|
|
|
feature branch and reviewed local HEAD, the fixed HTTPS Atlas origin, base
|
|
|
|
|
`main`, confirmation phrase, and one unique ephemeral ref. The command policy is
|
|
|
|
|
then pinned to that exact ref. It allows only its push, one exact draft-create
|
|
|
|
|
payload, cleanup of the exactly discovered PR number, and deletion of that ref.
|
hermes: add a fail-closed full-handoff acceptance harness
Decides whether the Hermes platform handoff is fit to release, and refuses
to round an absence of evidence up to a pass.
The harness is read-only by default and classifies 71 checks PASS / FAIL /
NOT_RUN / NOT_APPLICABLE. Any mandatory FAIL or NOT_RUN is NO_GO, and so is a
harness-level problem: an unreachable vantage, a catalog entry whose evidence
no longer exists, an expired deadline, or an evaluator that raised.
Evidence comes from two vantages that cannot cover for each other: an external
read-only operator kubeconfig, and the Hermes agent probing itself from inside
its own pod. Before any check runs, the harness asks each vantage who it is and
stops if they are the same principal, because dual-vantage evidence from one
identity is a restatement rather than a corroboration. `--as` is rejected for
every operator-side command and reachable only as the inner command of a
`kubectl exec`, so impersonation can never stand in for a real self-probe. A
deny check needs a live refused request, not only an authorization review.
Two safety properties are structural rather than conventional, enforced where
an argv becomes a subprocess: the default mode mutates nothing (mutating verbs
require a server dry run; there is deliberately no live TokenRequest probe,
because a successful one would mint a real credential), and no probe can pull a
credential value into a report (no vault/sops/curl, secrets readable only with
-o name, environment probes list names, shell only through frozen reviewed
templates). Captures are bounded before they are screened, and the rendered
report is re-screened before it is written.
Mutation lives behind a separate arming flag with an exact confirmation phrase,
a caller-supplied unique ref, a preflight that refuses a protected push target
before any network call, and a cleanup whose verification is itself mandatory.
A default run reports those four checks NOT_RUN.
The catalog is declarative so a reviewer reads what is asserted rather than how
it is plumbed, and so structural properties can be proven over every entry
before a run. Catalog drift surfaces as NOT_RUN, which stops the release.
docs/hermes_full_handoff_acceptance.md carries the merge order for PRs #14-#18
on top of the merged #13 baseline, the image build and Flux rollout, the
rollback point for each step, the go/no-go checklist, and the limits that are
asserted rather than exercised.
Validation: 295 handoff tests pass with 100% line coverage on all 15 new
modules; the full unit suite is 647 passed with two failures that reproduce
unchanged on origin/main; Ruff, py_compile, kustomize render, and a diff
credential screen are clean; a live read-only run against Atlas returns NO_GO
for the pre-merge cluster with no unscreened fields in the report.
2026-08-17 10:14:17 +00:00
|
|
|
|
2026-08-17 13:11:34 +00:00
|
|
|
The run reserves at least 300 seconds (up to 6000) for cleanup. It verifies the
|
fix(hermes): close the zero-evidence fail-open in absence checks
evaluate_names_absent returned PASS when its step exited 0 with no output,
so five mandatory checks - the ones asserting that provider API keys, forge
credentials, a cluster-admin binding, and shared coordinator state are
absent - could report a pass on no evidence and turn a NO_GO into a GO.
Both name rules now resolve their step through one guard in _line_step, so
zero observations are NOT_RUN. Regressions pin all five real catalog specs
plus both reachable silence paths: a POSIX pipeline whose status comes from
its last stage, and a drifted kubectl -o jsonpath. The pool claim projection
emits one <volume>=<claim> line per template volume so a volume without a
PVC still counts as an observation rather than reading as drift.
Also closes the review's reachable hardening and evidence defects:
- pin Gitea paths to atlas/titan-iac on an exact segment boundary and
reject relative segments, including percent-encoded ones
- forbid impersonation structurally in every mode and vantage; the inner
command of kubectl exec is re-checked rather than exempted, and
validate_catalog no longer guards only the operator vantage
- drop flux and helm from the binary allowlist; they had no pinned release
digest, so no allowlisted binary can now be admitted that the executor
would refuse to attest
- remove the inert --concurrency and --expect-telegram-sessions flags and
the dead concurrency bound; Telegram continuity stays mandatory
- read the ephemeral pull index page by page, treat the create response as
an authoritative source for the pull number, close every number either
source names, and surface residue_ref plus exact manual_cleanup commands
when creation is uncertain
- keep executable_path and executable_sha256 on unrecorded bulk-evidence
steps so withholding bytes never withholds binary attestation
- revert the repo-wide hygiene legacy-exception mechanism; the contract
change here is purely additive and the four pre-existing over-cap files
are left to the canonical contract change in PR #14/#15
- correct the runbook ruff format scope so the documented command passes
Split hermes_handoff_arming.py out of hermes_handoff_ephemeral.py to keep
both modules under the 500-line cap. All 16 handoff modules hold at least
95% line and branch coverage; the mutation gate is 13/13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 16:24:37 +00:00
|
|
|
pushed SHA, then identifies the pull request from two independent sources: the
|
|
|
|
|
create response itself, and the pull index read page by page to its last page
|
|
|
|
|
(50 per page, 20 pages maximum) rather than a single truncated first page. Either
|
|
|
|
|
source alone names a pull request this run definitely made, because preflight
|
|
|
|
|
proved the ephemeral ref did not exist. Every number found from either source is
|
|
|
|
|
registered and closed, so a failed or unusable index can no longer leave an
|
|
|
|
|
orphan open. A `GO` still requires exactly one open draft with the expected base,
|
|
|
|
|
head ref, base SHA, and head SHA, and the two sources must agree.
|
|
|
|
|
|
|
|
|
|
Cleanup requires successful PR close, successful branch deletion, a successful
|
|
|
|
|
empty `ls-remote`, and a strict re-read of the same closed PR. Malformed or
|
|
|
|
|
partial cleanup is `FAIL`; it is never inferred from an error. When anything is
|
|
|
|
|
uncertain the `draft-pull-request` evidence carries `residue_ref` and an exact
|
|
|
|
|
`manual_cleanup` command list — the `PATCH .../pulls/<n> --field state=closed`
|
|
|
|
|
calls for every number found, the `DELETE .../branches/<ref>` call, and, when no
|
|
|
|
|
number could be determined at all, the exact ref to sweep the index for by hand.
|
2026-08-17 13:11:34 +00:00
|
|
|
|
|
|
|
|
Force, force-with-lease, fan-out, protected refs, other repositories, other PR
|
|
|
|
|
numbers, arbitrary PATCH payloads, and repository administration remain denied.
|
|
|
|
|
|
|
|
|
|
## Dependency, merge, and conflict graph
|
|
|
|
|
|
|
|
|
|
Authoritative snapshot observed 2026-08-17 after refreshing Gitea and Git against
|
|
|
|
|
remote `main` `d8f2d818b9a552ea6c2d7fe86554be829bd5ffff`:
|
|
|
|
|
|
|
|
|
|
| PR | Head SHA | Draft | Base | Role |
|
|
|
|
|
| --- | --- | --- | --- | --- |
|
2026-08-17 16:27:28 +00:00
|
|
|
| #12 | `ed43dbaa7f094e1841aa760862e80015deb9d80c` | no | `main` | AI usage exporter repair |
|
2026-08-17 13:11:34 +00:00
|
|
|
| #14 | `8c6e3acdacb223f68b1716f0a74bd543e18890e6` | yes | `main` | bounded forge client and least-privilege RBAC |
|
|
|
|
|
| #15 | `73fefbb5d92ef07371a9a997f22d848acd2d06ed` | yes | `main` | result-decomposition reliability |
|
|
|
|
|
| #16 | `e8c26ecf85a0c802d378fffffb98b7df43b78242` | no | `main` | unprivileged image builder |
|
|
|
|
|
| #17 | `48cbe13ee50ce3fcb07cea3fe8d088cfed349e0c` | yes | `main` | three-lane placement |
|
2026-08-17 16:27:28 +00:00
|
|
|
| #18 | `689bcb6e48414854447988ba174143a307d308b9` | yes | `main` | distributed three-worker pool |
|
|
|
|
|
| #19 | `c808baff40a12a90f43c2e5e39b94d139f88ea29` | yes | `main` | head this repair replaces on the same branch |
|
2026-08-17 13:11:34 +00:00
|
|
|
| #20 | `a242dcc786576ae1a18000cb4941836ff610dff2` | yes | `main` | CLI process-group reaping |
|
|
|
|
|
|
|
|
|
|
The repaired #19 SHA cannot be embedded in its own commit without changing that
|
|
|
|
|
SHA. Resolve it from Gitea after push and require it as `--reviewed-head-sha`;
|
2026-08-17 16:27:28 +00:00
|
|
|
the PR handoff records the exact value. All eight heads in this table, including
|
|
|
|
|
the repaired #19, merge cleanly with `main` individually. Pairwise
|
|
|
|
|
`git merge-tree --write-tree` over all 28 combinations found conflicts for:
|
|
|
|
|
#12+#17, #12+#18, #12+#19, #14+#15, #14+#16, #14+#17, #14+#19, #14+#20, #15+#16,
|
|
|
|
|
#15+#17, #15+#19, #16+#19, #16+#20, #17+#18, and #17+#20. Gitea's
|
|
|
|
|
`mergeable=false` on a draft is not treated as conflict evidence.
|
|
|
|
|
|
|
|
|
|
This graph differs from the previous snapshot in exactly two ways. #12 and #18
|
|
|
|
|
both advanced, which adds #12+#18, #17+#18, and #12+#19. And #19 no longer
|
|
|
|
|
duplicates the repo-wide hygiene legacy-exception mechanism that #18 also
|
|
|
|
|
carries, which makes #18+#19 clean. Recompute before every integration step;
|
|
|
|
|
nothing here is durable.
|
2026-08-17 13:11:34 +00:00
|
|
|
|
|
|
|
|
Use this dependency order, refreshing every head and re-running merge-tree at
|
|
|
|
|
each step:
|
|
|
|
|
|
|
|
|
|
1. #14 first so the release never restores or prolongs cluster-admin authority.
|
|
|
|
|
2. #16, then #15, resolving their Dockerfile and quality-contract changes
|
|
|
|
|
additively. Never take one PR's quality contract wholesale.
|
|
|
|
|
3. #20 after #15; it depends on the CLI lifecycle work and must be rebased across
|
|
|
|
|
the current #14/#16 contract and helper changes.
|
|
|
|
|
4. #12 before #17, resolving their Deployment overlap once.
|
|
|
|
|
5. #17, then #18, after the authority, image, reliability, and reaping work.
|
|
|
|
|
6. #19 last as the integration/acceptance change, only after #12/#14-#18/#20 are
|
|
|
|
|
merged and their exact supplied heads are ancestors of refreshed `main`.
|
|
|
|
|
|
2026-08-17 16:27:28 +00:00
|
|
|
An exact post-repair merge-tree check reports #19 clean with current `main`, #17,
|
|
|
|
|
#18, and #20, and conflicting with #12, #14, #15, and #16 on these files:
|
|
|
|
|
|
|
|
|
|
| Pair | Overlapping files |
|
|
|
|
|
| --- | --- |
|
|
|
|
|
| #12+#19 | `testing/tests/test_hermes_auto_router.py` |
|
|
|
|
|
| #14+#19 | `testing/quality_contract.json`, `testing/quality_coverage.py`, `testing/tests/test_quality_coverage_helpers.py` |
|
|
|
|
|
| #15+#19 | `testing/tests/conftest.py` |
|
|
|
|
|
| #16+#19 | `testing/quality_contract.json` |
|
|
|
|
|
|
|
|
|
|
Every one of those overlaps is additive on the #19 side. Resolve the contract by
|
|
|
|
|
union — take every PR's managed modules, lint paths, tracked files, and branch
|
|
|
|
|
tracked files, and never take one PR's quality contract wholesale. #19 adds two
|
|
|
|
|
fixture-isolation fixtures to `conftest.py` and two lines to the auto-router test;
|
|
|
|
|
keep both sides. Preserve branch coverage, every tracked module, fixture
|
|
|
|
|
isolation, and all desired-state changes during conflict resolution.
|
|
|
|
|
|
|
|
|
|
Recompute this graph from the Gitea API immediately before integration; do not
|
|
|
|
|
infer conflict status from Gitea's draft `mergeable` field.
|
2026-08-17 13:12:36 +00:00
|
|
|
|
2026-08-17 13:11:34 +00:00
|
|
|
The #16/#19 integration must keep both the immutable digest and the source tag
|
|
|
|
|
(`newTag: git-<main-SHA>-build-<n>` plus the digest) in rendered image identity.
|
|
|
|
|
Digest-only rendering loses the attested build-source link and therefore
|
|
|
|
|
correctly remains `NO_GO` under the exact-lineage evaluator.
|
|
|
|
|
|
|
|
|
|
## Safe rollback guidance
|
|
|
|
|
|
|
|
|
|
Every rollback is a reviewed forward fix or selective Git revert followed by
|
|
|
|
|
the normal Flux workflow. Never restore `hermes-agent-cluster-admin`, even when
|
|
|
|
|
reverting unrelated portions of #14. If a rollback would widen RBAC, stop and
|
|
|
|
|
write a forward least-privilege fix instead.
|
|
|
|
|
|
|
|
|
|
Reverting #18 removes the StatefulSet desired state, but its
|
|
|
|
|
`volumeClaimTemplates` PVCs are retained by the Kubernetes default because no
|
|
|
|
|
delete retention policy is set. Do not claim Flux pruning deletes those PVCs.
|
|
|
|
|
Any PVC deletion is a separate, explicit, human-reviewed data-destruction task
|
|
|
|
|
with exact targets and recovery implications.
|
|
|
|
|
|
|
|
|
|
Image rollback means a reviewed commit restoring a previously attested digest;
|
|
|
|
|
never retag an immutable image. Do not use direct `kubectl` edits as a durable
|
|
|
|
|
rollback.
|
|
|
|
|
|
|
|
|
|
## Quality and release evidence
|
|
|
|
|
|
fix(hermes): close the zero-evidence fail-open in absence checks
evaluate_names_absent returned PASS when its step exited 0 with no output,
so five mandatory checks - the ones asserting that provider API keys, forge
credentials, a cluster-admin binding, and shared coordinator state are
absent - could report a pass on no evidence and turn a NO_GO into a GO.
Both name rules now resolve their step through one guard in _line_step, so
zero observations are NOT_RUN. Regressions pin all five real catalog specs
plus both reachable silence paths: a POSIX pipeline whose status comes from
its last stage, and a drifted kubectl -o jsonpath. The pool claim projection
emits one <volume>=<claim> line per template volume so a volume without a
PVC still counts as an observation rather than reading as drift.
Also closes the review's reachable hardening and evidence defects:
- pin Gitea paths to atlas/titan-iac on an exact segment boundary and
reject relative segments, including percent-encoded ones
- forbid impersonation structurally in every mode and vantage; the inner
command of kubectl exec is re-checked rather than exempted, and
validate_catalog no longer guards only the operator vantage
- drop flux and helm from the binary allowlist; they had no pinned release
digest, so no allowlisted binary can now be admitted that the executor
would refuse to attest
- remove the inert --concurrency and --expect-telegram-sessions flags and
the dead concurrency bound; Telegram continuity stays mandatory
- read the ephemeral pull index page by page, treat the create response as
an authoritative source for the pull number, close every number either
source names, and surface residue_ref plus exact manual_cleanup commands
when creation is uncertain
- keep executable_path and executable_sha256 on unrecorded bulk-evidence
steps so withholding bytes never withholds binary attestation
- revert the repo-wide hygiene legacy-exception mechanism; the contract
change here is purely additive and the four pre-existing over-cap files
are left to the canonical contract change in PR #14/#15
- correct the runbook ruff format scope so the documented command passes
Split hermes_handoff_arming.py out of hermes_handoff_ephemeral.py to keep
both modules under the 500-line cap. All 16 handoff modules hold at least
95% line and branch coverage; the mutation gate is 13/13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 16:24:37 +00:00
|
|
|
The canonical quality contract explicitly manages, lints, and tracks all 16
|
2026-08-17 13:11:34 +00:00
|
|
|
`scripts/ops/hermes_handoff_*.py` modules. It enforces at least 95% line and
|
|
|
|
|
branch coverage per tracked file and the 500-physical-line source cap. Contract
|
|
|
|
|
tests derive the module set so a future file cannot silently escape the gate.
|
fix(hermes): close the zero-evidence fail-open in absence checks
evaluate_names_absent returned PASS when its step exited 0 with no output,
so five mandatory checks - the ones asserting that provider API keys, forge
credentials, a cluster-admin binding, and shared coordinator state are
absent - could report a pass on no evidence and turn a NO_GO into a GO.
Both name rules now resolve their step through one guard in _line_step, so
zero observations are NOT_RUN. Regressions pin all five real catalog specs
plus both reachable silence paths: a POSIX pipeline whose status comes from
its last stage, and a drifted kubectl -o jsonpath. The pool claim projection
emits one <volume>=<claim> line per template volume so a volume without a
PVC still counts as an observation rather than reading as drift.
Also closes the review's reachable hardening and evidence defects:
- pin Gitea paths to atlas/titan-iac on an exact segment boundary and
reject relative segments, including percent-encoded ones
- forbid impersonation structurally in every mode and vantage; the inner
command of kubectl exec is re-checked rather than exempted, and
validate_catalog no longer guards only the operator vantage
- drop flux and helm from the binary allowlist; they had no pinned release
digest, so no allowlisted binary can now be admitted that the executor
would refuse to attest
- remove the inert --concurrency and --expect-telegram-sessions flags and
the dead concurrency bound; Telegram continuity stays mandatory
- read the ephemeral pull index page by page, treat the create response as
an authoritative source for the pull number, close every number either
source names, and surface residue_ref plus exact manual_cleanup commands
when creation is uncertain
- keep executable_path and executable_sha256 on unrecorded bulk-evidence
steps so withholding bytes never withholds binary attestation
- revert the repo-wide hygiene legacy-exception mechanism; the contract
change here is purely additive and the four pre-existing over-cap files
are left to the canonical contract change in PR #14/#15
- correct the runbook ruff format scope so the documented command passes
Split hermes_handoff_arming.py out of hermes_handoff_ephemeral.py to keep
both modules under the 500-line cap. All 16 handoff modules hold at least
95% line and branch coverage; the mutation gate is 13/13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 16:24:37 +00:00
|
|
|
This change is purely additive to the contract: it registers its own modules and
|
|
|
|
|
changes no repo-wide hygiene semantics.
|
|
|
|
|
|
|
|
|
|
The gate's `hygiene` step is red on `main` and stays red here. Exactly four
|
|
|
|
|
pre-existing test files exceed the 500-line cap —
|
|
|
|
|
`scripts/tests/test_dashboards_render_atlas.py` (516),
|
|
|
|
|
`testing/tests/test_hermes_chat_quality.py` (2242),
|
|
|
|
|
`testing/tests/test_hermes_cli_lanes.py` (1971), and
|
|
|
|
|
`testing/tests/test_hermes_coordinator.py` (510). All four are byte-identical at
|
|
|
|
|
merge-base `30259b52` and at current `origin/main`, none is touched by this
|
|
|
|
|
change, and the same four are the only issues a pristine `main` reports. Splitting
|
|
|
|
|
them, or grandfathering them, is a repo-wide contract decision that belongs to the
|
|
|
|
|
canonical contract change carried by PR #14/#15, not to this acceptance harness.
|
|
|
|
|
Every other gate step — docs, smell, unit, coverage — is green, and every file
|
|
|
|
|
this change adds or edits is under the cap.
|
2026-08-17 13:11:34 +00:00
|
|
|
|
|
|
|
|
```sh
|
|
|
|
|
python3 -m testing.quality_gate --profile local --build-dir build
|
|
|
|
|
python3 -m testing.quality_handoff_mutation
|
|
|
|
|
python3 -m ruff check --select F,B,SIM,C4,UP --ignore B017,UP015,UP035 \
|
fix(hermes): close the zero-evidence fail-open in absence checks
evaluate_names_absent returned PASS when its step exited 0 with no output,
so five mandatory checks - the ones asserting that provider API keys, forge
credentials, a cluster-admin binding, and shared coordinator state are
absent - could report a pass on no evidence and turn a NO_GO into a GO.
Both name rules now resolve their step through one guard in _line_step, so
zero observations are NOT_RUN. Regressions pin all five real catalog specs
plus both reachable silence paths: a POSIX pipeline whose status comes from
its last stage, and a drifted kubectl -o jsonpath. The pool claim projection
emits one <volume>=<claim> line per template volume so a volume without a
PVC still counts as an observation rather than reading as drift.
Also closes the review's reachable hardening and evidence defects:
- pin Gitea paths to atlas/titan-iac on an exact segment boundary and
reject relative segments, including percent-encoded ones
- forbid impersonation structurally in every mode and vantage; the inner
command of kubectl exec is re-checked rather than exempted, and
validate_catalog no longer guards only the operator vantage
- drop flux and helm from the binary allowlist; they had no pinned release
digest, so no allowlisted binary can now be admitted that the executor
would refuse to attest
- remove the inert --concurrency and --expect-telegram-sessions flags and
the dead concurrency bound; Telegram continuity stays mandatory
- read the ephemeral pull index page by page, treat the create response as
an authoritative source for the pull number, close every number either
source names, and surface residue_ref plus exact manual_cleanup commands
when creation is uncertain
- keep executable_path and executable_sha256 on unrecorded bulk-evidence
steps so withholding bytes never withholds binary attestation
- revert the repo-wide hygiene legacy-exception mechanism; the contract
change here is purely additive and the four pre-existing over-cap files
are left to the canonical contract change in PR #14/#15
- correct the runbook ruff format scope so the documented command passes
Split hermes_handoff_arming.py out of hermes_handoff_ephemeral.py to keep
both modules under the 500-line cap. All 16 handoff modules hold at least
95% line and branch coverage; the mutation gate is 13/13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 16:24:37 +00:00
|
|
|
scripts/ops/hermes_handoff_*.py testing/quality_coverage.py \
|
|
|
|
|
testing/quality_handoff_mutation.py testing/tests/test_hermes_handoff_*.py
|
2026-08-17 13:11:34 +00:00
|
|
|
python3 -m ruff format --check \
|
fix(hermes): close the zero-evidence fail-open in absence checks
evaluate_names_absent returned PASS when its step exited 0 with no output,
so five mandatory checks - the ones asserting that provider API keys, forge
credentials, a cluster-admin binding, and shared coordinator state are
absent - could report a pass on no evidence and turn a NO_GO into a GO.
Both name rules now resolve their step through one guard in _line_step, so
zero observations are NOT_RUN. Regressions pin all five real catalog specs
plus both reachable silence paths: a POSIX pipeline whose status comes from
its last stage, and a drifted kubectl -o jsonpath. The pool claim projection
emits one <volume>=<claim> line per template volume so a volume without a
PVC still counts as an observation rather than reading as drift.
Also closes the review's reachable hardening and evidence defects:
- pin Gitea paths to atlas/titan-iac on an exact segment boundary and
reject relative segments, including percent-encoded ones
- forbid impersonation structurally in every mode and vantage; the inner
command of kubectl exec is re-checked rather than exempted, and
validate_catalog no longer guards only the operator vantage
- drop flux and helm from the binary allowlist; they had no pinned release
digest, so no allowlisted binary can now be admitted that the executor
would refuse to attest
- remove the inert --concurrency and --expect-telegram-sessions flags and
the dead concurrency bound; Telegram continuity stays mandatory
- read the ephemeral pull index page by page, treat the create response as
an authoritative source for the pull number, close every number either
source names, and surface residue_ref plus exact manual_cleanup commands
when creation is uncertain
- keep executable_path and executable_sha256 on unrecorded bulk-evidence
steps so withholding bytes never withholds binary attestation
- revert the repo-wide hygiene legacy-exception mechanism; the contract
change here is purely additive and the four pre-existing over-cap files
are left to the canonical contract change in PR #14/#15
- correct the runbook ruff format scope so the documented command passes
Split hermes_handoff_arming.py out of hermes_handoff_ephemeral.py to keep
both modules under the 500-line cap. All 16 handoff modules hold at least
95% line and branch coverage; the mutation gate is 13/13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 16:24:37 +00:00
|
|
|
scripts/ops/hermes_handoff_*.py testing/quality_coverage.py \
|
|
|
|
|
testing/quality_handoff_mutation.py testing/tests/test_hermes_handoff_*.py
|
2026-08-17 13:11:34 +00:00
|
|
|
python3 -m compileall -q scripts/ops testing
|
|
|
|
|
kustomize build services/hermes | kubectl apply --dry-run=client --validate=false -f -
|
|
|
|
|
git diff --check origin/main...HEAD
|
hermes: add a fail-closed full-handoff acceptance harness
Decides whether the Hermes platform handoff is fit to release, and refuses
to round an absence of evidence up to a pass.
The harness is read-only by default and classifies 71 checks PASS / FAIL /
NOT_RUN / NOT_APPLICABLE. Any mandatory FAIL or NOT_RUN is NO_GO, and so is a
harness-level problem: an unreachable vantage, a catalog entry whose evidence
no longer exists, an expired deadline, or an evaluator that raised.
Evidence comes from two vantages that cannot cover for each other: an external
read-only operator kubeconfig, and the Hermes agent probing itself from inside
its own pod. Before any check runs, the harness asks each vantage who it is and
stops if they are the same principal, because dual-vantage evidence from one
identity is a restatement rather than a corroboration. `--as` is rejected for
every operator-side command and reachable only as the inner command of a
`kubectl exec`, so impersonation can never stand in for a real self-probe. A
deny check needs a live refused request, not only an authorization review.
Two safety properties are structural rather than conventional, enforced where
an argv becomes a subprocess: the default mode mutates nothing (mutating verbs
require a server dry run; there is deliberately no live TokenRequest probe,
because a successful one would mint a real credential), and no probe can pull a
credential value into a report (no vault/sops/curl, secrets readable only with
-o name, environment probes list names, shell only through frozen reviewed
templates). Captures are bounded before they are screened, and the rendered
report is re-screened before it is written.
Mutation lives behind a separate arming flag with an exact confirmation phrase,
a caller-supplied unique ref, a preflight that refuses a protected push target
before any network call, and a cleanup whose verification is itself mandatory.
A default run reports those four checks NOT_RUN.
The catalog is declarative so a reviewer reads what is asserted rather than how
it is plumbed, and so structural properties can be proven over every entry
before a run. Catalog drift surfaces as NOT_RUN, which stops the release.
docs/hermes_full_handoff_acceptance.md carries the merge order for PRs #14-#18
on top of the merged #13 baseline, the image build and Flux rollout, the
rollback point for each step, the go/no-go checklist, and the limits that are
asserted rather than exercised.
Validation: 295 handoff tests pass with 100% line coverage on all 15 new
modules; the full unit suite is 647 passed with two failures that reproduce
unchanged on origin/main; Ruff, py_compile, kustomize render, and a diff
credential screen are clean; a live read-only run against Atlas returns NO_GO
for the pre-merge cluster with no unscreened fields in the report.
2026-08-17 10:14:17 +00:00
|
|
|
```
|
|
|
|
|
|
fix(hermes): close the zero-evidence fail-open in absence checks
evaluate_names_absent returned PASS when its step exited 0 with no output,
so five mandatory checks - the ones asserting that provider API keys, forge
credentials, a cluster-admin binding, and shared coordinator state are
absent - could report a pass on no evidence and turn a NO_GO into a GO.
Both name rules now resolve their step through one guard in _line_step, so
zero observations are NOT_RUN. Regressions pin all five real catalog specs
plus both reachable silence paths: a POSIX pipeline whose status comes from
its last stage, and a drifted kubectl -o jsonpath. The pool claim projection
emits one <volume>=<claim> line per template volume so a volume without a
PVC still counts as an observation rather than reading as drift.
Also closes the review's reachable hardening and evidence defects:
- pin Gitea paths to atlas/titan-iac on an exact segment boundary and
reject relative segments, including percent-encoded ones
- forbid impersonation structurally in every mode and vantage; the inner
command of kubectl exec is re-checked rather than exempted, and
validate_catalog no longer guards only the operator vantage
- drop flux and helm from the binary allowlist; they had no pinned release
digest, so no allowlisted binary can now be admitted that the executor
would refuse to attest
- remove the inert --concurrency and --expect-telegram-sessions flags and
the dead concurrency bound; Telegram continuity stays mandatory
- read the ephemeral pull index page by page, treat the create response as
an authoritative source for the pull number, close every number either
source names, and surface residue_ref plus exact manual_cleanup commands
when creation is uncertain
- keep executable_path and executable_sha256 on unrecorded bulk-evidence
steps so withholding bytes never withholds binary attestation
- revert the repo-wide hygiene legacy-exception mechanism; the contract
change here is purely additive and the four pre-existing over-cap files
are left to the canonical contract change in PR #14/#15
- correct the runbook ruff format scope so the documented command passes
Split hermes_handoff_arming.py out of hermes_handoff_ephemeral.py to keep
both modules under the 500-line cap. All 16 handoff modules hold at least
95% line and branch coverage; the mutation gate is 13/13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 16:24:37 +00:00
|
|
|
The format scope names the files this change owns. A wider `testing/quality_*.py`
|
|
|
|
|
glob picks up `testing/quality_gate.py`, which is unmodified here and not
|
|
|
|
|
format-clean, so the documented command would fail for a reason this change did
|
|
|
|
|
not cause. `ruff format` is evidence, not a gate step: the canonical gate runs
|
|
|
|
|
`ruff check` only.
|
|
|
|
|
|
2026-08-17 13:11:34 +00:00
|
|
|
The deterministic mutation gate must kill every policy, redaction, verdict,
|
fix(hermes): close the zero-evidence fail-open in absence checks
evaluate_names_absent returned PASS when its step exited 0 with no output,
so five mandatory checks - the ones asserting that provider API keys, forge
credentials, a cluster-admin binding, and shared coordinator state are
absent - could report a pass on no evidence and turn a NO_GO into a GO.
Both name rules now resolve their step through one guard in _line_step, so
zero observations are NOT_RUN. Regressions pin all five real catalog specs
plus both reachable silence paths: a POSIX pipeline whose status comes from
its last stage, and a drifted kubectl -o jsonpath. The pool claim projection
emits one <volume>=<claim> line per template volume so a volume without a
PVC still counts as an observation rather than reading as drift.
Also closes the review's reachable hardening and evidence defects:
- pin Gitea paths to atlas/titan-iac on an exact segment boundary and
reject relative segments, including percent-encoded ones
- forbid impersonation structurally in every mode and vantage; the inner
command of kubectl exec is re-checked rather than exempted, and
validate_catalog no longer guards only the operator vantage
- drop flux and helm from the binary allowlist; they had no pinned release
digest, so no allowlisted binary can now be admitted that the executor
would refuse to attest
- remove the inert --concurrency and --expect-telegram-sessions flags and
the dead concurrency bound; Telegram continuity stays mandatory
- read the ephemeral pull index page by page, treat the create response as
an authoritative source for the pull number, close every number either
source names, and surface residue_ref plus exact manual_cleanup commands
when creation is uncertain
- keep executable_path and executable_sha256 on unrecorded bulk-evidence
steps so withholding bytes never withholds binary attestation
- revert the repo-wide hygiene legacy-exception mechanism; the contract
change here is purely additive and the four pre-existing over-cap files
are left to the canonical contract change in PR #14/#15
- correct the runbook ruff format scope so the documented command passes
Split hermes_handoff_arming.py out of hermes_handoff_ephemeral.py to keep
both modules under the 500-line cap. All 16 handoff modules hold at least
95% line and branch coverage; the mutation gate is 13/13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 16:24:37 +00:00
|
|
|
numeric, truncation, provenance, and cleanup mutant, including one that restores
|
|
|
|
|
the zero-evidence pass, one that restores the bare-prefix repository pin, one
|
|
|
|
|
that stops re-checking a `kubectl exec` inner command, and one that drops binary
|
|
|
|
|
attestation from an unrecorded step. The adversarial suite includes large
|
2026-08-17 13:11:34 +00:00
|
|
|
concurrent stdout/stderr, malformed JSON and bytes, zero state, partial failure,
|
|
|
|
|
absolute timeout, descendant-held pipes, malformed cleanup, ambiguous creates,
|
|
|
|
|
and no-subprocess/no-mutation spies. Render/client dry-run, exact conflict checks,
|
|
|
|
|
and bounded diff/full-tree secret scans must also pass.
|
|
|
|
|
|
|
|
|
|
The provider-health and Gitea-token-sensitive tests use hermetic fixtures. Their
|
|
|
|
|
result must be identical whether synthetic runtime files exist or are absent;
|
|
|
|
|
tests must never read the mounted production token.
|
|
|
|
|
|
|
|
|
|
## Release decision
|
|
|
|
|
|
|
|
|
|
The handoff remains `NO_GO` until all dependency heads are merged, the exact
|
|
|
|
|
lineage check passes from an external operator, both default and armed reports
|
|
|
|
|
are complete, the branch and PR cleanup is verified, the canonical and
|
|
|
|
|
adversarial gates are green, and a human has reviewed PR #19 plus the artifacts.
|
|
|
|
|
The harness never merges, deploys, reconciles, publishes, or approves anything.
|