38 Commits

Author SHA1 Message Date
jenkins
e3e40f4618 feat(monitoring): make Hermes triage email mean something
Every human_required escalation already files an issue in the failing
service's own repository, and mailing on each one made the inbox the loudest
and least informative output of the system. Replace the blanket alert with
two narrow ones: a repair that ran and failed, which is the only case where
the automation acted and left things no better, and an escalation still
untouched after six hours.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 00:26:34 -03:00
jenkins
363adedd75 feat(hermes-triage): human-required alert rule + Ariadne autotriage config
- vmalert: hermes-triage.yaml alert group (HermesTriageHumanRequired on
  ariadne_hermes_triage_incident{status="human_required"}, 1m interval),
  wire Alertmanager notifier into vmalert-atlas-availability
- Ariadne deployment: hermes autotriage env (observe mode: autotriage on,
  autoremediation OFF), 1m schedule, allowlist hermes-triage-demo only,
  HERMES_API_KEY from optional hermes-api-server-key Secret
2026-08-05 16:51:48 -03:00
jenkins
2691581cbc monitoring: publish availability outside query pool 2026-08-04 21:40:24 -03:00
jenkins
a8dda9815d monitoring: add daily availability rollups 2026-08-04 21:24:47 -03:00
jenkins
f724e72a96 monitoring: use request success for availability 2026-08-04 21:12:31 -03:00
jenkins
4906fb73d8 monitoring: measure gateway availability 2026-08-04 21:02:19 -03:00
jenkins
302fe78681 monitoring: preserve serving availability history 2026-08-04 18:39:36 -03:00
jenkins
bbab7330ce monitoring: avoid unstable volume hosts 2026-08-04 18:06:43 -03:00
jenkins
17b25c4429 monitoring: reload vmalert rules automatically 2026-08-04 12:24:38 -03:00
jenkins
b0599f4318 monitoring: prevent query pool starvation 2026-08-04 12:22:23 -03:00
jenkins
974955ac83 monitoring(testing): backfill category health rollups 2026-05-20 14:39:07 -03:00
jenkins
109698a2e3 monitoring(testing): attach branch labels to run rollups 2026-05-20 12:54:12 -03:00
jenkins
e380b65eb9 monitoring(testing): memoize dashboard freshness panels 2026-05-20 12:45:50 -03:00
jenkins
3a06d29387 monitoring(testing): record check health timelines 2026-05-20 12:14:49 -03:00
jenkins
b70afe2f03 monitoring(testing): memoize slow dashboard panels 2026-05-20 11:52:25 -03:00
jenkins
fe37f12e32 monitoring(testing): surface current gate health 2026-05-20 11:01:28 -03:00
jenkins
c75e0d1b88 monitoring(testing): roll up current test case state 2026-05-19 11:09:29 -03:00
jenkins
3d043424b4 monitoring(testing): filter test category rollups 2026-05-18 19:32:17 -03:00
jenkins
690c82e61b monitoring(vmalert): roll test pass-rate rule update 2026-05-17 15:32:24 -03:00
jenkins
3e8667d48a monitoring(testing): count failed test cases in pass-rate rollup 2026-05-17 15:27:21 -03:00
jenkins
c404a967d0 monitoring: attach build branch to quality rollups 2026-05-16 18:22:16 -03:00
jenkins
42ce51baad monitoring: make current gate rollup use latest checks 2026-05-16 18:07:21 -03:00
jenkins
7f209fbbc9 monitoring: refresh quality rollups every minute 2026-05-16 18:00:52 -03:00
jenkins
65298d7357 monitoring: keep quality check status without branch labels 2026-05-16 17:49:37 -03:00
jenkins
7dc03eefce monitoring: reload vmalert quality rules 2026-05-16 17:29:52 -03:00
jenkins
ad86195436 monitoring: add typhon and category test telemetry 2026-05-16 15:38:26 -03:00
jenkins
160c960345 monitoring: add lesavka category test telemetry 2026-05-16 12:07:57 -03:00
jenkins
4527f29e7e monitoring: clarify testing and gitops dashboards 2026-05-15 22:07:41 -03:00
jenkins
a2d5c9c83e monitoring(testing): use lane timelines for test health 2026-05-15 21:05:13 -03:00
jenkins
944a778c0a monitoring: clarify testing dashboard health trends 2026-05-15 19:52:46 -03:00
jenkins
6adbe457c4 monitoring: tune testing dashboard and gate rollups 2026-05-15 14:26:06 -03:00
jenkins
dad9e4e8f2 monitoring: ignore availability scrape gaps 2026-05-10 16:38:05 -03:00
jenkins
eb57c1fe0f monitoring: count post-start availability gaps 2026-05-10 16:21:47 -03:00
jenkins
e7213d9d1c monitoring: fill pre-telemetry availability 2026-05-10 16:13:13 -03:00
jenkins
7b656dbaeb monitoring: restart vmalert on rule changes 2026-05-10 15:50:50 -03:00
jenkins
01af181442 monitoring: schedule availability rollup deterministically 2026-05-10 15:49:32 -03:00
jenkins
192a36cf8a monitoring: fix vmalert remote write endpoint 2026-05-10 15:47:34 -03:00
jenkins
7f7dde01de monitoring: precompute atlas availability rollup 2026-05-10 15:40:12 -03:00