WIP: monitoring(titan): capacity guardrails for titan-20/21/22 audit (v2) #51

Draft

hermes-automation wants to merge 3 commits from feature/t_26da4c88-titan-capacity-guardrails-v2 into main

Author	SHA1	Message	Date
Hermes Agent	344c0eed5e	docs(titan): fold in sonarqube as a 4th live titan-22 placement 17c5f5093 landed on main mid-revision, moving sonarqube onto titan-22 with the same scavenger/CPU-only/no-GPU pattern as Jenkins/VictoriaMetrics/ Wger. Too recent (~4min runtime) for trend data, but healthy so far. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-23 14:56:43 +00:00
Hermes Agent	4f48a1d3de	docs(titan): fold in live Jenkins/VM/Wger placement onto titan-22 Brad placed Jenkins, VictoriaMetrics, and Wger onto titan-22 directly (main commits 271f3e8c3, 0ebab9d41) between this audit's first pass and PR #50 going up. Revalidate with fresh 3h evidence instead of shipping a report that describes a since-superseded snapshot: titan-22 headroom is materially unchanged (22-26% CPU p95, mem 77%+ available) with the three new residents, Jellyfin shows 0 restarts/throttling throughout, but no GPU/transcode activity occurred in the window so the transcode-under-load validation is still outstanding. Also flags VictoriaMetrics now being a single point of failure for titan-22 observability (hard-pinned, no fallback), and updates the board-comment record (§7) to match what was actually posted to t_39cf1905/t_60d2deb6. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-23 14:56:43 +00:00
Hermes Agent	2185a90825	monitoring(titan): add capacity guardrails for titan-20/21/22 audit Evidence-led capacity/QoS audit of titan-20 (Hermes LLM fallback/ classifier), titan-21 (STT/TTS), and titan-22 (Jellyfin media-primary, restored) for t_26da4c88. titan-20/21 are CPU-committed with no safe headroom (titan-20 at 227 MiB free memory at its 24h worst point); titan-22 has real idle CPU/RAM but its shared-GPU time-slicing has no VRAM/engine isolation, so no workload is relocated. Adds alerting for the sharpest gaps found (titan-20 memory exhaustion, titan-22 CPU/RAM/ GPU-VRAM pressure, Jellyfin CPU throttling, titan-21 CPU pressure) and two Atlas GPU dashboard panels (VRAM, NVENC/NVDEC utilization) so a future opportunistic-workload PR or a live transcode incident is visible without a promql session. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-23 14:56:43 +00:00

Author

SHA1

Message

Date

Hermes Agent

344c0eed5e

docs(titan): fold in sonarqube as a 4th live titan-22 placement

17c5f5093 landed on main mid-revision, moving sonarqube onto titan-22
with the same scavenger/CPU-only/no-GPU pattern as Jenkins/VictoriaMetrics/
Wger. Too recent (~4min runtime) for trend data, but healthy so far.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-08-23 14:56:43 +00:00

Hermes Agent

4f48a1d3de

docs(titan): fold in live Jenkins/VM/Wger placement onto titan-22

Brad placed Jenkins, VictoriaMetrics, and Wger onto titan-22 directly
(main commits 271f3e8c3, 0ebab9d41) between this audit's first pass and
PR #50 going up. Revalidate with fresh 3h evidence instead of shipping a
report that describes a since-superseded snapshot: titan-22 headroom is
materially unchanged (22-26% CPU p95, mem 77%+ available) with the three
new residents, Jellyfin shows 0 restarts/throttling throughout, but no
GPU/transcode activity occurred in the window so the transcode-under-load
validation is still outstanding. Also flags VictoriaMetrics now being a
single point of failure for titan-22 observability (hard-pinned, no
fallback), and updates the board-comment record (§7) to match what was
actually posted to t_39cf1905/t_60d2deb6.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-08-23 14:56:43 +00:00

Hermes Agent

2185a90825

monitoring(titan): add capacity guardrails for titan-20/21/22 audit

Evidence-led capacity/QoS audit of titan-20 (Hermes LLM fallback/
classifier), titan-21 (STT/TTS), and titan-22 (Jellyfin media-primary,
restored) for t_26da4c88. titan-20/21 are CPU-committed with no safe
headroom (titan-20 at 227 MiB free memory at its 24h worst point);
titan-22 has real idle CPU/RAM but its shared-GPU time-slicing has no
VRAM/engine isolation, so no workload is relocated. Adds alerting for
the sharpest gaps found (titan-20 memory exhaustion, titan-22 CPU/RAM/
GPU-VRAM pressure, Jellyfin CPU throttling, titan-21 CPU pressure) and
two Atlas GPU dashboard panels (VRAM, NVENC/NVDEC utilization) so a
future opportunistic-workload PR or a live transcode incident is
visible without a promql session.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-08-23 14:56:43 +00:00

WIP: monitoring(titan): capacity guardrails for titan-20/21/22 audit (v2) #51

3 Commits