bstein/titan-iac
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(ariadne): stop the kubelet killing a healthy triage pod

Browse Source 
The liveness probe used the default timeoutSeconds of 1. The auto-triage tick
runs every minute and spends most of it waiting on Jenkins, OpenSearch, Gitea
and Hermes, so against a 500m CPU limit /health occasionally answers in over a
second. Three of those and the container is killed, dropping triage ticks for
the length of a restart. Observed 11 times in 139 minutes, with the pod
sitting 1/2 Ready and restarting repeatedly.

Give both probes a 5s timeout and let liveness tolerate five failures, so a
busy tick is no longer mistaken for a hung process.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
jenkins 2026-08-06 05:20:14 -03:00
parent fbe61f6cb9
commit 3461a4e99d
1 changed files with 11 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
services/maintenance/apps/ariadne-deployment.yaml
View File

@ -565,15 +565,26 @@ spec:
limits:
cpu: 500m
memory: 512Mi
# timeoutSeconds defaults to 1, which this pod cannot honour. The
# auto-triage tick runs every minute and spends most of it waiting on
# Jenkins, OpenSearch, Gitea and Hermes; against a 500m CPU limit the
# event loop occasionally takes longer than a second to answer
# /health. Three of those in a row and the kubelet kills a container
# that is working perfectly well. Observed 11 times in 139 minutes,
# each one dropping triage ticks for the length of a restart.
livenessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3