From 1e67bab40a27fe56d388cae2e3e5299c47e78187 Mon Sep 17 00:00:00 2001 From: jenkins Date: Wed, 5 Aug 2026 23:36:58 -0300 Subject: [PATCH] docs: record the in-process repair timings and the classification-bleed fix Co-Authored-By: Claude Opus 5 --- knowledge/hermes-automated-triage-24h-plan.md | 17 +++++++++++++++++ .../hermes-automated-triage-24h-plan.md | 17 +++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/knowledge/hermes-automated-triage-24h-plan.md b/knowledge/hermes-automated-triage-24h-plan.md index 83fbae2b8..b7ffa6594 100644 --- a/knowledge/hermes-automated-triage-24h-plan.md +++ b/knowledge/hermes-automated-triage-24h-plan.md @@ -1017,6 +1017,14 @@ armed 20:58:10 -> build red 20:59:59 -> repair Job 21:01:14 second run: red 21:43:55 -> repair 21:44:39 -> green 21:50:45 -> resolved 21:51:08 ``` +After converting the fixture repair from a spawned Kubernetes Job to an +in-process API call (Ariadne 0.1.0-402), the automated leg roughly halved: + +```text +build red 02:20:03 -> fixture healthy 02:20:28 (25s) + -> rebuild triggered 02:20:41 -> green and resolved 02:21:07 (1m04s) +``` + Code proposal: red build 22:34:59 -> pull request open 22:35:32 (33s). Hermes diagnosis latency across all runs: 15-21s. @@ -1041,6 +1049,15 @@ Real homegrown service diagnoses (no action allowlisted, all escalated): job went green, the alert self-cleared through supersession. - `soteria` returned `unknown` at 0.58 — below the 0.85 threshold and with no matching action, so two independent gates blocked it. +- `bstein-dev-home/446` was classified `known_demo_fixture_failure` at 0.99, + even though build 445 of the same job had been classified correctly as + `sonarqube_node_runtime_missing`. The evidence gate refused it again, so + nothing ran, but the intermittency is the point: the same evidence produced + a demo-only classification on one run and the right one on another. The + triage prompt now interpolates the job under analysis and forbids the demo + classification and `repair_demo_fixture` on any job other than + `hermes-triage-demo` (Ariadne 0.1.0-403), removing the judgement call + rather than asking the model to make it correctly every time. ## Deviations from the plan diff --git a/services/comms/knowledge/hermes-automated-triage-24h-plan.md b/services/comms/knowledge/hermes-automated-triage-24h-plan.md index 83fbae2b8..b7ffa6594 100644 --- a/services/comms/knowledge/hermes-automated-triage-24h-plan.md +++ b/services/comms/knowledge/hermes-automated-triage-24h-plan.md @@ -1017,6 +1017,14 @@ armed 20:58:10 -> build red 20:59:59 -> repair Job 21:01:14 second run: red 21:43:55 -> repair 21:44:39 -> green 21:50:45 -> resolved 21:51:08 ``` +After converting the fixture repair from a spawned Kubernetes Job to an +in-process API call (Ariadne 0.1.0-402), the automated leg roughly halved: + +```text +build red 02:20:03 -> fixture healthy 02:20:28 (25s) + -> rebuild triggered 02:20:41 -> green and resolved 02:21:07 (1m04s) +``` + Code proposal: red build 22:34:59 -> pull request open 22:35:32 (33s). Hermes diagnosis latency across all runs: 15-21s. @@ -1041,6 +1049,15 @@ Real homegrown service diagnoses (no action allowlisted, all escalated): job went green, the alert self-cleared through supersession. - `soteria` returned `unknown` at 0.58 — below the 0.85 threshold and with no matching action, so two independent gates blocked it. +- `bstein-dev-home/446` was classified `known_demo_fixture_failure` at 0.99, + even though build 445 of the same job had been classified correctly as + `sonarqube_node_runtime_missing`. The evidence gate refused it again, so + nothing ran, but the intermittency is the point: the same evidence produced + a demo-only classification on one run and the right one on another. The + triage prompt now interpolates the job under analysis and forbids the demo + classification and `repair_demo_fixture` on any job other than + `hermes-triage-demo` (Ariadne 0.1.0-403), removing the judgement call + rather than asking the model to make it correctly every time. ## Deviations from the plan