From 3a1c5ea2dc609088d1b3d291aa5922d7a7bc4ac8 Mon Sep 17 00:00:00 2001 From: jenkins Date: Wed, 30 Sep 2026 11:11:04 -0500 Subject: [PATCH] hermes: record successful LAN recovery verification --- docs/hermes_suite_node_recovery_20260930.md | 33 +++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/docs/hermes_suite_node_recovery_20260930.md b/docs/hermes_suite_node_recovery_20260930.md index 5f925941..954d8cb6 100644 --- a/docs/hermes_suite_node_recovery_20260930.md +++ b/docs/hermes_suite_node_recovery_20260930.md @@ -42,3 +42,36 @@ be collected. Stop new submissions, reconcile the two metadata ledgers with owner/idempotency conflicts rejected, then restore the node selector and volume reference through Git. Never switch back to an older ledger while accepting jobs, and retain both PVCs during recovery. Do not restart an active inference job. + +## Verification + +Recovery manifest commit: `0574eb590cd2a1ddae8a12e3535353be0c3699f5`. +Flux applied this revision; Hermes and its required dependencies reported Ready. +The planner reported one ready/available replica on titan-24, with zero restarts. +Both the original and recovery PVCs remained Bound. + +Kustomize rendering and client dry runs passed for Hermes and the Flux root. +The changed planner resources also passed a server-side dry run. This kubectl +version rejects combining `--server-side` with `--dry-run=client`; those modes +were checked separately. Flux diff showed only the expected placement and +volume changes in Hermes. + +From the existing LAN jump host, HTTPS used `worker.bstein.dev` with an explicit +connection to `192.168.22.50:443`, certificate verification enabled, proxies +disabled, and redirects rejected. Operational-token capabilities and health +returned 200; missing and invalid credentials returned 401; `/api/pull` returned +404. Tokens were read privately from the injected credentials and not printed. + +One synthetic 14-case job completed through the same HTTPS path: + +- Job: `8766db2f378f408491361b04c9e3cf13`. +- Serialized request: 9847 bytes. +- Server duration: 55.431 seconds; client duration: 56.708 seconds. +- Runtime model: `claude-opus-5-5`. +- Three validated model passes; zero retries. +- Nine natural families and nine final tasks; maximum group size two. +- Exact alias coverage passed. Repeated submission with the same key returned + the same job rather than launching duplicate inference. + +No real suite was rerun. Laptop connectivity itself was not tested from here. +Titan-22's fault and its inaccessible historical job outcome remain unresolved.