5636 Commits

Author SHA1 Message Date
Hermes Agent
344c0eed5e docs(titan): fold in sonarqube as a 4th live titan-22 placement
17c5f5093 landed on main mid-revision, moving sonarqube onto titan-22
with the same scavenger/CPU-only/no-GPU pattern as Jenkins/VictoriaMetrics/
Wger. Too recent (~4min runtime) for trend data, but healthy so far.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 14:56:43 +00:00
Hermes Agent
4f48a1d3de docs(titan): fold in live Jenkins/VM/Wger placement onto titan-22
Brad placed Jenkins, VictoriaMetrics, and Wger onto titan-22 directly
(main commits 271f3e8c3, 0ebab9d41) between this audit's first pass and
PR #50 going up. Revalidate with fresh 3h evidence instead of shipping a
report that describes a since-superseded snapshot: titan-22 headroom is
materially unchanged (22-26% CPU p95, mem 77%+ available) with the three
new residents, Jellyfin shows 0 restarts/throttling throughout, but no
GPU/transcode activity occurred in the window so the transcode-under-load
validation is still outstanding. Also flags VictoriaMetrics now being a
single point of failure for titan-22 observability (hard-pinned, no
fallback), and updates the board-comment record (§7) to match what was
actually posted to t_39cf1905/t_60d2deb6.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 14:56:43 +00:00
Hermes Agent
2185a90825 monitoring(titan): add capacity guardrails for titan-20/21/22 audit
Evidence-led capacity/QoS audit of titan-20 (Hermes LLM fallback/
classifier), titan-21 (STT/TTS), and titan-22 (Jellyfin media-primary,
restored) for t_26da4c88. titan-20/21 are CPU-committed with no safe
headroom (titan-20 at 227 MiB free memory at its 24h worst point);
titan-22 has real idle CPU/RAM but its shared-GPU time-slicing has no
VRAM/engine isolation, so no workload is relocated. Adds alerting for
the sharpest gaps found (titan-20 memory exhaustion, titan-22 CPU/RAM/
GPU-VRAM pressure, Jellyfin CPU throttling, titan-21 CPU pressure) and
two Atlas GPU dashboard panels (VRAM, NVENC/NVDEC utilization) so a
future opportunistic-workload PR or a live transcode incident is
visible without a promql session.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 14:56:43 +00:00
jenkins
17c5f5093a placement(quality): use titan-22 spare capacity 2026-08-23 11:50:39 -03:00
jenkins
42795f3d61 cleanup(hermes): remove Veles observer binding 2026-08-23 11:41:33 -03:00
jenkins
0ebab9d412 placement(wger): use preemptible titan-22 capacity 2026-08-23 11:23:09 -03:00
jenkins
e22e4793bb health(wger): replace expensive application probes 2026-08-23 11:21:19 -03:00
jenkins
435e258e45 rollout(hermes): activate Claude health hysteresis 2026-08-23 11:19:48 -03:00
jenkins
6b378fd410 fix(hermes): preserve proven Claude authentication 2026-08-23 11:18:18 -03:00
jenkins
0682b2b813 resources(finance): right-size Actual Budget memory 2026-08-23 11:07:33 -03:00
jenkins
1e66a0bedf fix(hermes): tolerate worker lease handoff 2026-08-23 11:01:20 -03:00
jenkins
e9e355f25a resources: release Pi scheduler headroom 2026-08-23 10:53:57 -03:00
jenkins
6fabb61844 resources(comms): right-size idle Synapse requests 2026-08-23 10:50:53 -03:00
jenkins
b7be05427e fix(hermes): place arm64 agent on titan-08 2026-08-23 10:45:51 -03:00
jenkins
f1739e7108 fix(hermes): keep sessions off storage nodes 2026-08-23 10:43:42 -03:00
jenkins
1e6eda8633 fix: converge load-spread rollouts 2026-08-23 10:41:01 -03:00
jenkins
271f3e8c32 ops: spread saturated node workloads 2026-08-23 10:37:01 -03:00
jenkins
a019ecd556 monitoring(ai): compare provider quota windows 2026-08-23 09:48:25 -03:00
ced9a24cd9 Merge pull request 'Hermes WebUI release lane v2' (#48) from feature/t_8cbe6a55-hermes-webui-release-v2 into main
Reviewed-on: atlas/titan-iac#48
Reviewed-by: bstein <bstein@noreply.scm.bstein.dev>
2026-08-23 12:19:10 +00:00
58e1a29b2b Merge branch 'main' into feature/t_8cbe6a55-hermes-webui-release-v2
Some checks failed
Tests / Declarative: Post Actions failed: 39, skipped: 19, passed: 2782
2026-08-23 12:18:58 +00:00
flux-bot
51515932fc chore(bstein-dev-home): automated image update 2026-08-23 09:58:39 +00:00
flux-bot
1eaaf57abd chore(bstein-dev-home): automated image update 2026-08-23 09:57:02 +00:00
jenkins
fe7832efaf hermes: retain dual-provider quota health
Some checks failed
Tests / Declarative: Post Actions failed: 39, skipped: 19, passed: 2757
2026-08-23 02:52:42 -03:00
Hermes Agent
083005e44f fix: verify served Hermes PWA identity 2026-08-23 04:42:08 +00:00
Hermes Agent
99a9de936b fix: verify WebUI OCI source revision 2026-08-23 03:52:03 +00:00
Hermes Agent
735a2b240d Add reviewed Hermes WebUI release lane 2026-08-23 03:52:03 +00:00
jenkins
81b3e6b992 hermes: expose Claude remaining quota 2026-08-23 00:42:39 -03:00
jenkins
d036062519 hermes: stabilize AI quota collection 2026-08-23 00:14:58 -03:00
jenkins
3da6a10174 hermes: remove provider-biased auto fallbacks 2026-08-22 23:54:40 -03:00
jenkins
a7e09f9719 monitoring(ai): measure Claude routed usage 2026-08-22 23:38:40 -03:00
jenkins
69ad76bebc hermes: keep distributed workers on Claude 2026-08-22 23:02:15 -03:00
flux-bot
5136a0026e chore(maintenance): automated image update 2026-08-23 01:59:50 +00:00
flux-bot
42a86687c4 chore(maintenance): automated image update 2026-08-23 01:58:38 +00:00
flux-bot
049c39b339 chore(maintenance): automated image update 2026-08-23 01:54:50 +00:00
jenkins
8acb99d100 hermes: make Claude setup-token access durable 2026-08-22 22:52:28 -03:00
flux-bot
5e0b0d9cd6 chore(maintenance): automated image update 2026-08-23 01:49:36 +00:00
jenkins
b8294f8e90 longhorn: reduce Jellyfin policy churn 2026-08-22 22:18:03 -03:00
jenkins
eec373dd56 jellyfin: run media service on titan-22 2026-08-22 22:10:46 -03:00
jenkins
5430c9ac01 jellyfin: prepare titan-22 media host 2026-08-22 22:06:39 -03:00
jenkins
76fe550871 node(titan-22): remove completed canaries 2026-08-22 21:22:33 -03:00
jenkins
21264a71c8 node(titan-22): return worker to service 2026-08-22 21:21:03 -03:00
jenkins
091a787fc7 node(titan-22): persist Realtek firmware 2026-08-22 20:55:47 -03:00
jenkins
074f988731 node(titan-22): migrate k3s to USB ethernet 2026-08-22 20:42:51 -03:00
jenkins
db0cb86ca6 hermes: stabilize Claude access health probes 2026-08-22 18:07:58 -03:00
jenkins
9bffc077e8 hermes: avoid recursive home ownership rollout
Some checks failed
Tests / Declarative: Post Actions failed: 40, skipped: 19, passed: 2746
2026-08-22 17:38:42 -03:00
jenkins
bd50a7e4ad hermes: persist Claude subscription access 2026-08-22 17:32:30 -03:00
jenkins
9bf41dc0de hermes: decouple node maintenance health 2026-08-22 16:39:44 -03:00
jenkins
fd42e892f7 hermes: avoid flapping worker node 2026-08-22 16:17:33 -03:00
jenkins
fdf53b264b hermes: fit owner pod on healthy workers 2026-08-22 15:55:30 -03:00
jenkins
45a7a4dded platform: right-size Vault sidecar requests 2026-08-22 15:48:41 -03:00