ai(hermes): include operator proof sprint

This commit is contained in:
jenkins 2026-08-02 04:01:40 -03:00
parent 4991493d3b
commit 96edd28b53

View File

@ -0,0 +1,82 @@
# Atlas two-hour proof sprint
The sprint establishes usable, defensible evidence for the current Hermes
claims. It does not award mastery and it cannot retroactively prove historical
OpenClaw use. Brad performs the work; Hermes gives one task at a time, checks
the evidence, and withholds later answers until Brad commits to a result.
## 015 minutes — control surface and request path
1. Have Brad select `openai-codex/gpt-5.4` for the training session.
2. In the web UI, have him locate Chat, Sessions, Files, Models, Logs, Skills,
Plugins, MCP, Channels, Webhooks, Cron, Profiles, Config, Keys, System, and
Documentation.
3. Have him explain browser → operator agent → model gate → Ollama, including
Codex fallback and the separate consumer instance.
Evidence: Brad can identify persistent, shared, and instance-local state and
can name the GPU owner check without changing it.
## 1535 minutes — permissions and safety
1. Have Brad inspect the operator ServiceAccount, ClusterRole, NetworkPolicy,
and approval deny list using read-only commands.
2. Have him produce an allow/deny matrix for pod/log reads, Secret reads,
exec, deployment mutation, Flux reads, Flux reconcile, and internal
evidence endpoints.
3. Require him to explain why prompt instructions and skills are not the
primary security boundary.
Evidence: correct identity/RBAC/network/approval layering with no Secret values
read and no mutation attempted.
## 3575 minutes — perform the claimed triage path
Invoke `$triage-titan-test-failures`. Brad must:
1. check Ariadne diagnosis and bundle freshness;
2. identify one current or retained failed suite/build;
3. correlate Jenkins log/artifact evidence;
4. correlate Git/Flux revision and Kubernetes environment health;
5. query the relevant Pushgateway/VictoriaMetrics quality signal and identify
the matching Grafana context;
6. produce Finding, Confidence, Evidence, Likely cause, Blast radius, Next
checks, Repo-side fix, and Approval required.
Hermes may suggest a read-only command after Brad explains why it is needed.
Do not let the built-in diagnosis replace Brad's own evidence correlation.
Evidence: an 8/10 or better result under the triage rubric with no invented
facts and no unacknowledged mutation.
## 75105 minutes — build the reusable workflow
1. Have Brad inspect the mounted `triage-titan-test-failures` skill and explain
its trigger, fixed procedure, variable evidence, and security boundary.
2. Have Brad choose one genuinely repeated sub-workflow from the triage.
3. Guide him to create a new skill in the operator's writable skill area. Do
not edit the mounted Flux skill and do not write the skill for him.
4. Test one expected trigger and one adversarial/non-trigger case.
5. Have Brad make one improvement based on the test results.
Evidence: Brad can find the skill in the Skills UI, explain every instruction,
and show a passing and adversarial evaluation.
## 105120 minutes — supervised proposal and claim audit
1. Have Brad turn the triage into the smallest repo-side proposal without
applying it.
2. Have him identify every step requiring approval: file edit, commit/push,
pipeline trigger, Flux reconcile, credential/environment change, or manual
backfill.
3. Audit the interview answer sentence by sentence using `mastery-rubric.md`.
4. Offer to record progress only after Brad approves the write.
Minimum honest outcome:
- current Hermes deployment and supervised triage use are demonstrated;
- the established Atlas automation path and read-only boundary are explained;
- one reusable writable skill is created, tested, and improved by Brad;
- autonomous production mutation is not claimed;
- historical OpenClaw replacement remains unproven unless independent retained
evidence is produced.