jenkins
1894078b62
fix(scripts): dashboard renderer wrote outside the repo after the layout move
...
ROOT still used parents[1], which resolved to scripts/ once the renderer
moved into scripts/render/. Every --build run wrote a phantom
scripts/services/monitoring tree and silently left the real dashboards
untouched. Points at the repo root again and removes the stray tree.
Also adds Hermes triage panels to the Atlas Testing dashboard: open
escalations awaiting a human, automated actions succeeded, Hermes
diagnosis latency, actions by result, and incident state by job.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 21:41:02 -03:00
jenkins
84ff647898
fix(monitoring): Alertmanager must HELO with a FQDN for Mailu
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 21:33:11 -03:00
jenkins
664e436575
feat(hermes-triage): real-repo patch proposals + Alertmanager email escalation
...
- Ariadne: per-repo code config for metis, lesavka, soteria,
bstein-dev-home and ariadne, each with its own base branch, source path
prefixes and file suffixes so a proposal can only touch that repo's
source tree
- Alertmanager: the only receiver was an empty "default", so every alert
fired into a void. HermesTriageHumanRequired now routes to an email
receiver via Mailu's in-cluster local-domain relay, with resolved
notices; scoped to service=hermes-triage so nothing else mails yet.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 21:30:34 -03:00
jenkins
f4e2da317c
feat(hermes-triage): human-required alert rule + Ariadne autotriage config
...
- vmalert: hermes-triage.yaml alert group (HermesTriageHumanRequired on
ariadne_hermes_triage_incident{status="human_required"}, 1m interval),
wire Alertmanager notifier into vmalert-atlas-availability
- Ariadne deployment: hermes autotriage env (observe mode: autotriage on,
autoremediation OFF), 1m schedule, allowlist hermes-triage-demo only,
HERMES_API_KEY from optional hermes-api-server-key Secret
2026-08-05 16:51:48 -03:00
jenkins
c4b4e8a404
Merge origin/main into layout migration
...
Reconciles the services layout migration with ~991 upstream commits:
- Remote content wins for cassandra/cassandra-auth, monitoring dashboards,
vmalert availability rules, veles, vault auth script, dashboard render
script and tests (request-v4 availability definition)
- Layout paths win for structure: keycloak/bstein-dev-home job dirs use
bootstrap-jobs/validation-jobs; cassandra realm jobs live in
cassandra-auth (removed keycloak duplicates)
- Union: applications CR list gains hermes-chat and cassandra
image-automation
- Fixed post-migration paths in dashboard test module loader and
hermes-access job header
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 16:26:36 -03:00
jenkins
8f45f47e60
refactor: restructure services layout, retire oceanus, add aether scaffolding
...
- Move flat service manifests into structured subdirs (apps/, bootstrap-jobs/,
repair-jobs/, migration-jobs/, validation-jobs/, node-ops/, networking/)
- Retire oneoffs/ directories across services
- Remove oceanus cluster and its host roles; add aether cluster + terraform scaffolding
- Reorganize scripts/ into ops/, render/, sync/, manual-tests/
- Add Makefile with render/validate/test/flux targets and repo-structure tests
- Update flux-system application CRs to the new paths
- Add hermes-automated-triage-24h-plan knowledge doc (+ comms mirror)
- Refresh knowledge catalogs, dashboards, vmalert rules, quality contract
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 16:21:36 -03:00
jenkins
798e3fee84
Revert "monitoring(network): organize Traefik traffic lanes"
...
This reverts commit eeec8b72dc885950dd116d6dc2ad6574c1cbe6b1.
2026-08-05 12:06:23 -03:00
jenkins
48253cdaf0
monitoring(network): organize Traefik traffic lanes
2026-08-05 12:02:51 -03:00
jenkins
24308f4870
monitoring: remove legacy availability series
2026-08-04 22:49:48 -03:00
jenkins
a2564f8611
monitoring: retain daily availability through retries
2026-08-04 21:47:30 -03:00
jenkins
ba4b871a62
monitoring: publish availability outside query pool
2026-08-04 21:40:24 -03:00
jenkins
af10f7aadc
monitoring: add daily availability rollups
2026-08-04 21:24:47 -03:00
jenkins
5f692ecd3d
monitoring: add compact availability fallback
2026-08-04 21:14:06 -03:00
jenkins
83696d46d2
monitoring: use request success for availability
2026-08-04 21:12:31 -03:00
jenkins
6b1d6388db
monitoring: measure gateway availability
2026-08-04 21:02:19 -03:00
jenkins
026db898fc
monitoring: preserve serving availability history
2026-08-04 18:39:36 -03:00
jenkins
8d16b790af
monitoring: fall back to live serving state
2026-08-04 18:16:13 -03:00
jenkins
5fabd1a84b
monitoring: avoid unstable volume hosts
2026-08-04 18:06:43 -03:00
jenkins
e9b4404e02
monitoring: render test category zero state
2026-08-04 12:34:08 -03:00
jenkins
7c6e7f9736
monitoring: reload vmalert rules automatically
2026-08-04 12:24:38 -03:00
jenkins
e354f3f83a
monitoring: prevent query pool starvation
2026-08-04 12:22:23 -03:00
jenkins
1c070d96d4
monitoring(grafana): use Texas timezone
2026-08-03 15:58:16 -03:00
jenkins
8e7327a7ef
fix(monitoring): reload provisioned Grafana rules
2026-08-03 04:05:33 -03:00
jenkins
a00175b376
feat(hermes): add actionable Atlas triage skills
2026-08-03 03:58:37 -03:00
jenkins
4dab530baf
monitoring(nodes): clamp CPU charts to physical range
2026-08-02 19:36:44 -03:00
jenkins
f2328f12e4
monitoring(gpu): attribute Jetson activity by allocation
2026-08-02 04:26:33 -03:00
jenkins
b1ecfe96e4
ai(hermes): add operator guide and current GPU shares
2026-08-02 03:59:32 -03:00
jenkins
df953359b9
monitoring(gpu): use process-level pod attribution
2026-08-02 03:31:02 -03:00
jenkins
3255936aeb
monitoring(gpu): report time-weighted namespace usage
2026-08-02 03:08:20 -03:00
jenkins
a155847915
feat(cassandra): add migration Grafana rollup
2026-07-25 00:29:03 -03:00
jenkins
48773efbb1
monitoring: filter node dashboards to real nodes
2026-07-14 20:24:45 -03:00
jenkins
c8707c002d
monitoring: collapse duplicate Typhon climate series
2026-07-14 19:01:53 -03:00
jenkins
618fd01cc5
monitoring: scope endpoint node relabel to node-exporter
2026-07-14 18:57:51 -03:00
jenkins
921edc257c
monitoring: relabel pod-network node-exporter names
2026-07-14 18:30:43 -03:00
jenkins
e03598caa2
monitoring: move node-exporter host port
2026-07-14 18:23:52 -03:00
jenkins
4f4a5f99b7
monitoring: avoid node-exporter host port collisions
2026-07-14 18:18:03 -03:00
jenkins
bea20b9c97
monitoring: track titan-23 and pin tiny apps to workers
2026-07-14 18:14:38 -03:00
jenkins
6d200d79ae
maintenance: expire completed one-off jobs
2026-07-14 17:10:18 -03:00
jenkins
8d3402beb7
monitoring: give victoria metrics recovery headroom
2026-06-18 22:23:48 -03:00
jenkins
e7ec333fb0
recovery(ananke): keep flux holds and place metrics on longhorn nodes
2026-06-18 22:08:14 -03:00
jenkins
b395fb6abb
monitoring: keep nvidia exporter off oceanus
2026-06-09 00:55:11 -03:00
jenkins
1d20fb35d2
veles: stage atlas infrastructure
2026-06-09 00:46:46 -03:00
jenkins
2e854f7b5b
monitoring(testing): cap history panel ranges
2026-06-05 13:22:29 -03:00
jenkins
0a031b8d13
monitoring(testing): cap expensive dashboard queries
2026-06-05 13:15:12 -03:00
jenkins
255b05bf44
monitoring(testing): restore lesavka suite visibility
2026-06-05 01:04:56 -03:00
jenkins
56a29fce03
monitoring(testing): clean up dashboard health signals
2026-06-04 16:09:08 -03:00
jenkins
04175d33ab
monitoring(gpu): show activity share by namespace
2026-05-22 04:22:51 -03:00
jenkins
250a850871
monitoring(gpu): count monitored GPU pool devices
2026-05-22 03:23:36 -03:00
jenkins
69584c8a60
monitoring: keep quality probe on worker nodes
2026-05-22 03:16:01 -03:00
jenkins
491da8a7f5
monitoring(gpu): add pool utilization counters
2026-05-22 03:09:10 -03:00
