253 Commits

Author SHA1 Message Date
96b1ceeb97 monitoring: stabilize alert queries 2026-01-29 13:07:55 -03:00
d286d169a0 monitoring: fix GPU share attribution 2026-01-28 19:08:53 -03:00
454c7fbd20 monitoring: de-dupe ariadne schedule alert 2026-01-28 18:45:15 -03:00
a20f3cc0ce monitoring: avoid ariadne alert title conflict 2026-01-28 14:02:12 -03:00
78c72a71a2 monitoring: disable legacy cron alert 2026-01-28 13:58:28 -03:00
26e35ffbaf monitoring: restart grafana to reload alerts 2026-01-28 13:53:33 -03:00
fb2feec1b5 monitoring: reuse maint-cron uid for ariadne alert 2026-01-28 13:52:12 -03:00
f94926a387 monitoring: restart grafana for ariadne alerts 2026-01-28 13:49:41 -03:00
44788b3132 monitoring: alert on ariadne schedules 2026-01-28 13:47:54 -03:00
cfa2b4c08b monitoring: fix grafana pod annotation indent 2026-01-28 12:37:42 -03:00
ee3543e70d monitoring: restart grafana after alert fix 2026-01-28 12:32:56 -03:00
780cdd6c1b monitoring: fix postmark alert metrics 2026-01-28 12:31:33 -03:00
244578cc01 chore: organize one-off jobs 2026-01-28 01:48:32 -03:00
b34f2abefd monitoring: fix grafana alert exec state 2026-01-27 23:34:11 -03:00
9409c037c9 monitoring: restart grafana for alerting reload 2026-01-27 23:29:46 -03:00
c5a7eece35 monitoring: tune cpu and maintenance alerts 2026-01-27 23:23:42 -03:00
ca7a08e791 monitoring: fix grafana smtp from address 2026-01-27 22:28:37 -03:00
029e4d4ca6 monitoring: send grafana alerts via postmark 2026-01-27 22:00:19 -03:00
38c8d08ab4 monitoring: fix gpu idle label 2026-01-27 21:46:58 -03:00
ba16f5119b monitoring: unify gpu namespace usage 2026-01-27 21:43:37 -03:00
51bf01a8fd monitoring: keep idle label in gpu share 2026-01-27 18:44:58 -03:00
1b04e6cb00 monitoring: fix gpu idle share 2026-01-27 17:51:13 -03:00
5f32dff73b monitoring: fix tegrastats regexes 2026-01-27 16:44:00 -03:00
dfb295e5f0 monitoring: expose jetson scrape line length 2026-01-27 16:38:09 -03:00
a7f3d49fea monitoring: read tegrastats per scrape 2026-01-27 16:34:31 -03:00
246ed6617e monitoring: read jetson stats on demand 2026-01-27 16:27:45 -03:00
1951291090 monitoring: refresh jetson stats on scrape 2026-01-27 16:23:23 -03:00
62a423f32c monitoring: fix jetson gpu metrics 2026-01-27 16:19:54 -03:00
9ea338b121 monitoring: restart jetson exporter 2026-01-26 22:51:41 -03:00
0331e7ea99 monitoring: fix jetson metrics newlines 2026-01-26 22:50:33 -03:00
1616994b19 monitoring: unify jetson gpu metrics 2026-01-26 22:26:24 -03:00
72bd22e912 monitoring: map dcgm to shared gpu resources 2026-01-26 20:58:06 -03:00
b0abb9bd6e ariadne: reduce comms noise, fix gpu labels 2026-01-26 20:54:33 -03:00
a988af3262 monitoring: alert on VM outage 2026-01-23 11:51:28 -03:00
ce5b1d1353 monitoring: add postgres metrics and update overview 2026-01-22 18:23:26 -03:00
d509dfaa22 ops: restore portal/ariadne and add postgres panels 2026-01-22 15:23:23 -03:00
4721d44a33 monitoring: enforce sorted job lists 2026-01-21 15:12:53 -03:00
db4c3b7c51 monitoring: tighten jobs/overview ordering 2026-01-21 15:01:02 -03:00
b0996e9a4f monitoring: refine jobs/overview panels 2026-01-21 14:31:11 -03:00
8b35ab0292 monitoring: refresh jobs dashboards 2026-01-21 13:37:36 -03:00
2e407e1962 monitoring: reschedule grafana user dedupe 2026-01-21 12:31:54 -03:00
5ae6b4b00c monitoring: harden grafana user dedupe 2026-01-21 12:30:08 -03:00
ae1fd5b661 monitoring: fix grafana user dedupe job 2026-01-21 12:25:53 -03:00
4e65f02fba monitoring: prepopulate vault for dedupe job 2026-01-21 12:18:57 -03:00
88de0f7cee monitoring: wire vault sa for dedupe job 2026-01-21 12:16:26 -03:00
08716c6be6 monitoring: use python dedupe job 2026-01-21 12:15:03 -03:00
a0caeb407c monitoring: dedupe grafana user via api 2026-01-21 12:11:28 -03:00
6eeb551239 monitoring: add grafana user dedupe job 2026-01-21 12:08:23 -03:00
98b063f2dd grafana: allow email-based oauth user lookup 2026-01-21 11:45:11 -03:00
698b2fd96b monitoring: refresh testing dashboard 2026-01-21 11:29:48 -03:00