bstein/titan-iac
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(runbook): Claude primary, hung-build escalation, narrowed alerting

Browse Source 
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
jenkins 2026-08-06 05:15:37 -03:00
parent 913f5953ab
commit dbef8d6244
2 changed files with 36 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
knowledge/runbooks/hermes-triage-demo.md
View File

@ -15,13 +15,28 @@ You arm the failure (SEED_FAILURE=true)
-> Ariadne calls the Hermes Agent API (/v1/runs) with -> Ariadne calls the Hermes Agent API (/v1/runs) with
$triage-titan-test-failures $triage-titan-test-failures
-> Hermes returns schema-valid diagnosis + requested_action -> Hermes returns schema-valid diagnosis + requested_action
-> Ariadne authorizes (nine gates) and, if remediation is enabled, -> Ariadne authorizes (twelve gates) and, if remediation is enabled,
creates Job hermes-demo-repair-<build> (writes "healthy") patches the fixture ConfigMap back to "healthy" in process
-> Ariadne triggers ONE rebuild with SEED_FAILURE=false -> Ariadne triggers ONE rebuild with SEED_FAILURE=false
-> rebuild passes -> incident resolved -> rebuild passes -> incident resolved
Anything else -> human_required metric -> vmalert HermesTriageHumanRequired Anything else -> issue in the service repo + human_required metric
``` ```
The model behind Hermes is `anthropic/claude-opus-5` as of 2026-08-06, with
`openai-codex/gpt-5.6-terra` as first fallback and a local `gpt-oss:20b`
behind that, so an expired Anthropic credential degrades rather than stops.
Two behaviours worth knowing before you demo, because both look like nothing
happening:
- **A build that never finishes** is escalated once it passes
`ARIADNE_HERMES_HUNG_BUILD_MINUTES` (default 45). No model is consulted -
the console is still being written - so the issue says only that the build
overran and is holding an agent slot.
- **Alerting no longer fires on every escalation.** The issue in the service
repository is the durable artifact. Email now means either a repair ran and
failed, or an escalation has sat untouched for six hours.
## Arming the demo ## Arming the demo
Jenkins UI: `https://ci.bstein.dev/job/hermes-triage-demo/` → *Build with Jenkins UI: `https://ci.bstein.dev/job/hermes-triage-demo/` → *Build with

21
services/comms/knowledge/hermes-triage-demo.md
View File

@ -15,13 +15,28 @@ You arm the failure (SEED_FAILURE=true)
-> Ariadne calls the Hermes Agent API (/v1/runs) with -> Ariadne calls the Hermes Agent API (/v1/runs) with
$triage-titan-test-failures $triage-titan-test-failures
-> Hermes returns schema-valid diagnosis + requested_action -> Hermes returns schema-valid diagnosis + requested_action
-> Ariadne authorizes (nine gates) and, if remediation is enabled, -> Ariadne authorizes (twelve gates) and, if remediation is enabled,
creates Job hermes-demo-repair-<build> (writes "healthy") patches the fixture ConfigMap back to "healthy" in process
-> Ariadne triggers ONE rebuild with SEED_FAILURE=false -> Ariadne triggers ONE rebuild with SEED_FAILURE=false
-> rebuild passes -> incident resolved -> rebuild passes -> incident resolved
Anything else -> human_required metric -> vmalert HermesTriageHumanRequired Anything else -> issue in the service repo + human_required metric
``` ```
The model behind Hermes is `anthropic/claude-opus-5` as of 2026-08-06, with
`openai-codex/gpt-5.6-terra` as first fallback and a local `gpt-oss:20b`
behind that, so an expired Anthropic credential degrades rather than stops.
Two behaviours worth knowing before you demo, because both look like nothing
happening:
- **A build that never finishes** is escalated once it passes
`ARIADNE_HERMES_HUNG_BUILD_MINUTES` (default 45). No model is consulted -
the console is still being written - so the issue says only that the build
overran and is holding an agent slot.
- **Alerting no longer fires on every escalation.** The issue in the service
repository is the durable artifact. Email now means either a repair ran and
failed, or an escalation has sat untouched for six hours.
## Arming the demo ## Arming the demo
Jenkins UI: `https://ci.bstein.dev/job/hermes-triage-demo/` → *Build with Jenkins UI: `https://ci.bstein.dev/job/hermes-triage-demo/` → *Build with