From 3461a4e99db8db5f5398e727ad53ada17045edf4 Mon Sep 17 00:00:00 2001 From: jenkins Date: Thu, 6 Aug 2026 05:20:14 -0300 Subject: [PATCH] fix(ariadne): stop the kubelet killing a healthy triage pod The liveness probe used the default timeoutSeconds of 1. The auto-triage tick runs every minute and spends most of it waiting on Jenkins, OpenSearch, Gitea and Hermes, so against a 500m CPU limit /health occasionally answers in over a second. Three of those and the container is killed, dropping triage ticks for the length of a restart. Observed 11 times in 139 minutes, with the pod sitting 1/2 Ready and restarting repeatedly. Give both probes a 5s timeout and let liveness tolerate five failures, so a busy tick is no longer mistaken for a hung process. Co-Authored-By: Claude Opus 5 --- services/maintenance/apps/ariadne-deployment.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/services/maintenance/apps/ariadne-deployment.yaml b/services/maintenance/apps/ariadne-deployment.yaml index aeaf194a9..218e1aeec 100644 --- a/services/maintenance/apps/ariadne-deployment.yaml +++ b/services/maintenance/apps/ariadne-deployment.yaml @@ -565,15 +565,26 @@ spec: limits: cpu: 500m memory: 512Mi + # timeoutSeconds defaults to 1, which this pod cannot honour. The + # auto-triage tick runs every minute and spends most of it waiting on + # Jenkins, OpenSearch, Gitea and Hermes; against a 500m CPU limit the + # event loop occasionally takes longer than a second to answer + # /health. Three of those in a row and the kubelet kills a container + # that is working perfectly well. Observed 11 times in 139 minutes, + # each one dropping triage ticks for the length of a restart. livenessProbe: httpGet: path: /health port: http initialDelaySeconds: 10 periodSeconds: 10 + timeoutSeconds: 5 + failureThreshold: 5 readinessProbe: httpGet: path: /health port: http initialDelaySeconds: 5 periodSeconds: 10 + timeoutSeconds: 5 + failureThreshold: 3