A build whose agent never started is a distinct failure from one that lost a
connection mid-run: retrying can work, but when the pool is already full of
stuck pods the retry queues behind them and fails identically. Clearing first
is what makes the retry worth making. The clear is Ariadne's existing
scheduled pod cleanup, which only removes pods that have already succeeded or
failed, so nothing running is touched. This is the failure behind lesavka's
open issue and behind two stalled demo runs tonight.
Critically, all three remediations are now described in the triage prompt.
They were wired in Ariadne but absent from what Hermes is told, so Hermes
could never have requested them - the allowlist would have advertised
capability that could not fire. A test now asserts every allowlisted action id
appears in the prompt, so the two cannot drift apart again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Second entry in the action registry, proving it is a real extension point.
No cluster mutation: the action is one Jenkins rebuild.
- hermes_infra_signals: reviewable marker set across DNS/connectivity,
image pull, upstream 5xx and agent-channel loss; Ariadne independently
confirms a marker in the evidence before any retry, and records which
marker justified it. "no space left on device" is deliberately excluded
because a retry lands on the same full volume.
- decision: classification -> action registry (action_classifications),
falling back to the previous single-classification behavior
- repair: retry_build posts to /build for unparameterized real jobs and
buildWithParameters for the fixture demo job
- orchestrator: retry path records requested/accepted/executed and moves
the incident to awaiting_rebuild so the existing success path resolves
it; one action per incident still enforced, so a retry cannot loop
- events layer split out of the orchestrator to stay under the LOC cap
Motivated by real failures tonight: pip DNS resolution and a Gitea 443
connect timeout, plus live incident metis/271 (SCM checkout timeout).
30 new tests; 368 pass in the hermes suite.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
