2 Commits

Author SHA1 Message Date
codex
ee7b243106 feat(hermes): clear stuck agent pods, and teach Hermes the new remediations
All checks were successful
Tests / Declarative: Post Actions passed: 1257
A build whose agent never started is a distinct failure from one that lost a
connection mid-run: retrying can work, but when the pool is already full of
stuck pods the retry queues behind them and fails identically. Clearing first
is what makes the retry worth making. The clear is Ariadne's existing
scheduled pod cleanup, which only removes pods that have already succeeded or
failed, so nothing running is touched. This is the failure behind lesavka's
open issue and behind two stalled demo runs tonight.

Critically, all three remediations are now described in the triage prompt.
They were wired in Ariadne but absent from what Hermes is told, so Hermes
could never have requested them - the allowlist would have advertised
capability that could not fire. A test now asserts every allowlisted action id
appears in the prompt, so the two cannot drift apart again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 21:04:38 -03:00
codex
ca2324ab95 feat(hermes): reclaim exhausted workspace storage, and stop hung builds
All checks were successful
Tests / Declarative: Post Actions passed: 1251
Two real actions, both for failures that interrupted this project while it was
being built, and both backed by code Ariadne already had.

reclaim_workspace_storage closes a gap left open deliberately: 'no space left
on device' was excluded from the transient retry because a rebuild lands on
the same full volume and either fails identically or hides a capacity problem.
Reclaiming first makes the retry meaningful. The reclaim is the existing
scheduled cleanup, with its own deletion budget, so no new capability is
granted - it is only reachable from triage now. Its signature requires both a
storage marker and a workspace hint, because a full disk elsewhere in the
cluster is a different failure that reclaiming Jenkins workspaces would not
address.

Hung-build detection previously filed an issue and left the build running,
holding one of five Jenkins agent slots and starving every other job - the
actual harm. Ariadne now stops it as well, which is reversible: the job can
simply be built again.

The action executors move to their own module. They are the only code in
triage that changes anything outside Ariadne and should be reviewable as one
unit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 20:55:35 -03:00