577 Commits

Author SHA1 Message Date
jenkins
36c7e76e56 nodes: document verified administration and remaining recovery work 2026-10-04 00:37:36 -05:00
jenkins
c3f892fd0e nodes: retire stale log hooks and reserve Ariadne memory 2026-10-04 00:22:46 -05:00
jenkins
1bd8ae3197 nodes: repair credential audit and preserve Vault records 2026-10-04 00:09:20 -05:00
jenkins
b0bfb6a948 backup: schedule application recovery copies and document verified repairs 2026-10-03 01:34:57 -05:00
jenkins
3eaada682c backup: add native application database recovery bundles 2026-10-03 01:12:43 -05:00
jenkins
0780c93f69 monitoring: expose datastore backup freshness and missing copies 2026-10-03 00:46:38 -05:00
jenkins
80aff49855 recovery: schedule native datastore backups with LAN replica 2026-10-03 00:19:34 -05:00
jenkins
9255a64b23 hermes: recover suite passes and adapt large-suite analysis 2026-09-30 01:52:19 -05:00
jenkins
cef01b0d72 hermes: adapt suite review capacity and report admission failures 2026-09-30 00:58:42 -05:00
jenkins
83446c23dc fix(hermes): allow explicit one-hour suite job budgets 2026-09-29 23:01:55 -05:00
jenkins
6178265f70 fix(hermes): retain safe CLI failure categories 2026-09-29 16:45:53 -05:00
jenkins
ce3025fb9c hermes: select xhigh reasoning for large suite jobs 2026-09-29 16:05:30 -05:00
jenkins
a281c99634 hermes: raise suite planning reasoning effort to high 2026-09-29 15:57:56 -05:00
jenkins
206c61993d hermes: upgrade suite planner for Claude 5.5 models 2026-09-29 15:50:36 -05:00
jenkins
9c327c6fd0 hermes: require one structured assignment per suite case 2026-09-29 14:10:03 -05:00
jenkins
15c05cb7fb hermes: report suite progress and extend approved job bounds 2026-09-29 13:53:46 -05:00
jenkins
7af4f4f206 hermes: reconcile whole-suite proposals and cap tasks at five cases 2026-09-29 13:29:10 -05:00
jenkins
2d762776c1 hermes: retain safe CLI diagnostics and allow format repair headroom 2026-09-29 11:43:02 -05:00
jenkins
299b9ed16d hermes: scope approved Claude access to an operational credential 2026-09-29 08:33:35 -05:00
jenkins
5af516f9c2 hermes: pin account-listed Claude context variant and bound ingress 2026-09-29 08:09:48 -05:00
jenkins
d7d2d5430b hermes: verify native planning runtime before readiness 2026-09-29 07:56:36 -05:00
jenkins
20bf855159 hermes: harden native LAN inference and document its contract 2026-09-28 18:58:00 -05:00
jenkins
279c363835 ai: add scoped planning adapter and fix structured reasoning calls 2026-09-28 18:39:44 -05:00
jenkins
8a3f53082f ai: keep invalid planning results out of reusable cache 2026-09-28 18:10:16 -05:00
jenkins
e9c901f177 ai: add curl access for the private batch API 2026-09-28 18:07:53 -05:00
jenkins
ff64e27aec docs: record local planning pilot and export boundaries 2026-09-28 18:07:06 -05:00
jenkins
721e013576 ai: add private batch client and planning pilot contracts 2026-09-28 18:06:22 -05:00
jenkins
713cb8f80f hermes: serve authenticated local inference on the LAN 2026-09-28 17:10:12 -05:00
jenkins
0a51ffe4a7 hermes: resume failed PR publications with trusted bounded retries 2026-09-13 19:11:35 -05:00
jenkins
87c0a8b201 fix(monitoring): bound Titan test metric labels 2026-09-02 23:50:39 -03:00
jenkins
0f32df9aee docs: follow local atlas-iac checkout path 2026-09-01 21:08:18 -03:00
jenkins
8d1302765f gitea: migrate sources to titan/atlas-iac 2026-09-01 20:43:50 -03:00
jenkins
db446c9244 monitoring(ai): alert before Claude quota auth expires 2026-08-25 19:16:01 -03:00
jenkins
342677dde7 monitoring(ai): preserve quota data across rollouts 2026-08-25 18:33:18 -03:00
jenkins
c4eb872690 monitoring(ai): show Claude Fable weekly quota 2026-08-24 14:13:35 -03:00
jenkins
a019ecd556 monitoring(ai): compare provider quota windows 2026-08-23 09:48:25 -03:00
jenkins
d036062519 hermes: stabilize AI quota collection 2026-08-23 00:14:58 -03:00
jenkins
a7e09f9719 monitoring(ai): measure Claude routed usage 2026-08-22 23:38:40 -03:00
jenkins
21264a71c8 node(titan-22): return worker to service 2026-08-22 21:21:03 -03:00
jenkins
bd50a7e4ad hermes: persist Claude subscription access 2026-08-22 17:32:30 -03:00
ed98278981 Merge branch 'main' into feature/hermes-domain-rename-agent-worker-v2
Some checks failed
Tests / Declarative: Post Actions failed: 40, skipped: 19, passed: 2743
2026-08-21 23:15:16 +00:00
Hermes Agent
94106bf252 refactor(hermes): rename the owner agent host to worker.bstein.dev
Introduce worker.bstein.dev as the canonical hostname for the owner-only
Hermes coordinator, previously agent.hermes.bstein.dev.

The rename is additive, matching the shape #38 restored for chat and triage.
CoreDNS, both agent Ingresses and the hermes-sites certificate now serve BOTH
names, so merging this cannot take away the endpoint the operator uses to
reach the coordinator. Retiring agent.hermes.bstein.dev is a separate,
separately scheduled change. No redirect middleware is added.

What switches to the new host:
- HERMES_DASHBOARD_PUBLIC_URL and the oauth2-proxy --redirect-url
- the Keycloak agent proxy rootUrl
- operator docs, skills, the ZAP baseline target and the triage monitor default

What stays dual-homed until retirement:
- CoreDNS hosts entry, both agent Ingress rules, certificate SANs
- API_SERVER_CORS_ORIGINS (now a comma-separated pair)
- the Keycloak redirect URIs, web origins and post-logout origins, so a
  rollback only needs the oauth2-proxy --redirect-url reverted and does not
  require re-running the ensure job

The agent client passes its legacy origin through the optional fourth argument
#38 added to ensure_proxy_client, so no second mechanism is introduced. The
immutable ensure Job goes -11 -> -12 because #38 already consumed -11 and that
run has completed; without a further bump this change would never be applied.
Login on the new host fails until the -12 Job completes.

Because the session and CSRF cookies use the __Host- prefix they are bound to
one origin, so a fresh login must start on worker.bstein.dev and existing
sessions do not carry over -- re-login is required after rollout.

#38's public-host continuity test now covers the agent proxy's dual origins
rather than asserting the agent surface was untouched by the rename.

Knowledge catalogs and diagrams regenerated with `make knowledge`.
2026-08-21 10:29:46 +00:00
Hermes Agent
2862594c62 fix(monitoring): measure Atlas availability honestly across telemetry gaps
The 2026-08-18 metrics-storage outage exposed two defects in the
availability pipeline that distorted the figure in opposite directions at
once.

The Overview panel fell back to a live one-hour Traefik ratio whenever the
yearly rollup sample went stale for 48h, and rendered it under the same
"365d" title. When the rollup stopped publishing on 2026-08-18 the panel
quietly swapped a 365-day measurement for a 60-minute one and read 99.74%
instead of the recorded 99.95%. The fallback is removed: a stale rollup now
renders no value, and a new atlas-availability-rollup-stale alert pages at
26h, well before the panel goes blank at 48h.

The yearly ratio also silently excluded the 34-hour telemetry gap, because
missing days contribute zero requests and zero failures. Absent data was
read as "nothing happened" — had Atlas genuinely been down in that window,
the figure would still have said 99.95%. Availability keeps its
measured-days-only definition, which is correct, but coverage is now
published alongside it and shown in a new panel, so a telemetry gap lowers
disclosed coverage instead of vanishing. The title reads "365d window" to
stop implying 365 days of data exist; request-v4 begins 2026-05-01.

The rollup job reported healthy runs across a day and a half of lost
publishes: a read-only VictoriaMetrics accepts an import and discards it.
It now reads each sample back and fails loudly when the write did not
survive.

Not addressed here: availability is still measured from inside the platform
via Traefik counters, so it cannot distinguish "Atlas down" from "telemetry
down", and misses failures that never reach Traefik (DNS, TLS, node dead).
An external synthetic prober is the real fix and needs a hosting decision.
2026-08-20 02:03:43 +00:00
Hermes Agent
c4eac8ceee fix(monitoring): keep VictoriaMetrics writable 2026-08-19 10:51:44 +00:00
jenkins
b6ae6225f6 hermes: source handoff forge evidence through the scm broker
The acceptance harness pinned a forge client that has never existed in any
commit or pod (/opt/coordinator/gitea_api.py, digest f0943db4..., GIT/POST
grammar, an askpass helper). Every Gitea-backed check was therefore
unrunnable as merged. Point the harness at the credential-isolated SCM
broker client that actually ships in the agent pod.

- policy: GITEA_CLIENT=/opt/scm/gitea_api.py; trust /opt/scm/ instead of
  the phantom /opt/coordinator/; admit the client's real grammar
  (`read <api-path>`, exactly one path) with the same atlas/titan-iac pin
  and dot-segment rejection; bare HTTP methods are refused in every mode.
  The armed POST/PATCH/DELETE windows remain but are documented as
  deferred: the deployed client cannot execute them.
- exec: pin the client digest to the sha256 of
  services/hermes/scm-common/scripts/gitea_api.py — the exact file the
  hermes-scm-boundary-v2 ConfigMap mounts at /opt/scm/gitea_api.py — so
  the pin is derivable from merged source and equal to the deployed
  client. gitea_api.py gains a narrow /api/v1/user identity read in
  _authorize_read (see below), so the pin is the NEW source hash
  76efd16dedbeb74425b12fbbdbfaa391854771292077e0463bf22706855ae6dc.
  Drop the dangling GIT_ASKPASS (no helper exists; broker git needs
  none) and swap /opt/coordinator for /opt/scm in SAFE_PATH.
- checks: all forge/baseline/lineage probes use (client, "read", path).
  The SELF-vantage identity checks now truthfully assert the *broker's*
  forge identity (the only one the platform can exercise) is not an
  administrator and holds push-scoped, non-administrative repository
  authority; the administrative-route check asserts the broker read
  allowlist's live refusal of branch_protections. The remote-main step
  keeps `origin` (the broker remote exists only in pool workspaces and
  the broker origin is cluster-local); its rationale now tells the
  operator to ensure origin fetchability.
- gitea_api.py/_authorize_read: allow exactly `/api/v1/user` (no query,
  no sibling routes) as operation "identity" so the harness can prove
  the broker identity is not an administrator. The broker imports the
  same module, so one reviewed edit covers both sides of the boundary.
- rules: DENIAL_MARKERS now match the client's real refusal lines
  ("SCM broker request failed with HTTP 400/403" and the no-credential
  rejection) and drop "gitea api returned http 403", which the client
  never emits; a broker 404 is deliberately not denial evidence.
- ephemeral: index/verification reads use the real grammar; manual
  cleanup guidance now says close/delete require operator forge
  credentials (the client exposes no mutation besides create-draft);
  armed mode is documented as deferred until the probes are rebuilt on
  the broker's bounded mutation surface.
- docs: broker vantage/evidence section, operator prerequisites (broker
  healthy, no /vault/secrets/gitea-token anywhere on the harness path,
  current ConfigMap mount, operator-side client + origin fetchability),
  armed-mode deferral.
- tests: read-grammar accepted / GET refused in every mode, /opt/scm
  attestation pin proven equal to the merged source digest, real
  denial-marker matching, /api/v1/user identity route bounds; the
  repository-pin mutant probe speaks the new grammar. Full handoff +
  gitea + broker families pass (952 tests), mutation gate 13/13, per-file
  line+branch coverage >=95%, all touched sources within the 500-line cap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 18:21:38 -03:00
jenkins
a5cafddd45 hermes: accept merged lineage in handoff acceptance
The reviewed PR stack is now merged into main, so an open draft PR #19 is
no longer proof that the reviewed code is what runs. The mandatory
release.exact-lineage-is-running check now requires the merged terminal
state instead: PR #19 closed with merged=true, base main, the existing
feature ref, and the exact reviewed head, plus a new merge-ancestry step
that proves the reviewed head is an ancestor of the pinned origin/main
via git merge-base --is-ancestor. An open PR, a PR closed without
merging, a mismatched head, or a head that is not a proven ancestor of
main still fails closed; an undecidable ancestry probe is NOT_RUN. The
recorded pre-merge base SHA is no longer compared against current main.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 16:28:25 -03:00
jenkins
943ecfd15b Merge remote-tracking branch 'origin/feature/hermes-distributed-worker-pool' into feature/hermes-full-handoff-acceptance
# Conflicts:
#	ci/scripts/semgrep_report.py
#	testing/quality_contract.json
#	testing/quality_coverage.py
#	testing/tests/conftest.py
#	testing/tests/test_hermes_auto_router.py
#	testing/tests/test_quality_contract.py
#	testing/tests/test_quality_coverage_helpers.py
#	testing/tests/test_semgrep_report.py
2026-08-18 06:28:18 -03:00
jenkins
8036101f0b Merge remote-tracking branch 'origin/feature/hermes-review-goal-semantics' into feature/hermes-distributed-worker-pool
# Conflicts:
#	scripts/tests/test_dashboards_render_atlas_drilldowns.py
#	scripts/tests/test_dashboards_render_jobs.py
#	services/hermes/scm-common/scripts/scm_broker.py
#	services/hermes/scripts/cli_lane_dispatch.py
#	services/hermes/scripts/cli_lane_execution.py
#	testing/quality_contract.json
#	testing/tests/test_hermes_agent_access.py
#	testing/tests/test_hermes_agent_security.py
#	testing/tests/test_hermes_chat_config.py
#	testing/tests/test_hermes_chat_images.py
#	testing/tests/test_hermes_chat_provider_auth.py
#	testing/tests/test_hermes_chat_quality.py
#	testing/tests/test_hermes_chat_support.py
#	testing/tests/test_hermes_chat_voice.py
#	testing/tests/test_hermes_cli_finalization_edges.py
#	testing/tests/test_hermes_cli_foundation_coverage.py
#	testing/tests/test_hermes_cli_lanes_configuration.py
#	testing/tests/test_hermes_cli_recovery_edges.py
#	testing/tests/test_hermes_cli_retention_edges.py
#	testing/tests/test_hermes_coordinator.py
#	testing/tests/test_hermes_coordinator_boards.py
#	testing/tests/test_hermes_coordinator_support.py
2026-08-18 01:43:39 -03:00
jenkins
a3077224e0 Merge origin/main (#14 + #16) into PR #15 decomposition reliability
Reconcile two independent test/gate reorganizations:
- Gate/semgrep/mailu: keep main's #16 dual-metric implementation.
- quality_contract.json: union #16 image-builder + #14 scm/node + #15 cli_lane.
- agent-deployment.yaml: keep #14 gitea removal + #16 image-build-token + #15 probe.
- Test splits: main's chat/coordinator/agent organization is authoritative;
  drop #15's redundant competing splits and #14's stale cli-lane duplicates;
  keep #15's cli-lane decomposition suite and port the execution-safety test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 00:42:53 -03:00
jenkins
9d42ff33f5 Merge origin/main (#16 image-builder) into hermes-safe-gitea-pr
Resolve 8 conflicts, uniting #16's daemonless image-release lane with
#14's SCM broker + node-audit boundary. Nothing dropped from either side.

- quality_coverage.py / test_quality_coverage_helpers.py: take main's
  dual-metric gate (line+branch enforced per file at minimum_percent);
  drop #14's now-dead branch_tracked_files/minimum_branch_percent keys.
- quality_contract.json: union all lists (managed_modules, lint_paths,
  coverage_sources, tracked_files, hygiene globs) so #14's scm-broker,
  git_pack_objects, receive_pack_scan, deadline_http and node_polkit_audit
  are branch-checked alongside #16's image-builder modules.
- Vault auth (vault_k8s_auth_configure.sh): coexist both role sets. Keep
  #16's hermes-switchyard split + hermes-jenkins-token-seed; keep #14's
  hermes-scm-broker role. Preserve #14's security property: hermes-agent
  no longer holds developer-gitea (broker role carries it).
- agent-deployment.yaml / stage_runtime_access.py: keep #16's
  jenkins-image-build-token injection/staging; keep #14's removal of the
  agent's gitea-token/gitea-username.
- Bump vault-k8s-auth-hermes job -9 -> -10 (and its health check + test)
  so the merged auth config re-applies over any -9 already in-cluster.
- flux hermes dependsOn: union jenkins + hermes-scm-broker + observer-rbac.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 21:07:06 -03:00