titan/atlas-iac
3
0
Fork 0
Code Issues Pull Requests 9 Actions Packages Projects Releases Wiki Activity

docs(titan): fold in sonarqube as a 4th live titan-22 placement

Browse Source 
17c5f5093 landed on main mid-revision, moving sonarqube onto titan-22
with the same scavenger/CPU-only/no-GPU pattern as Jenkins/VictoriaMetrics/
Wger. Too recent (~4min runtime) for trend data, but healthy so far.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Hermes Agent 2026-08-23 14:56:35 +00:00
parent 4f48a1d3de
commit 344c0eed5e
1 changed files with 16 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
docs/titan_20_21_22_capacity_optimization.md
View File

@ -64,6 +64,7 @@ usually has room. This matters for the Jenkins finding in §5.
| titan-22 | `jenkins` (added live, see §4a) | **scavenger (-10)** | 0.1 / 1.5 | 1Gi / 3Gi | none |
| titan-22 | `victoria-metrics-single-server` (added live, see §4a) | **scavenger (-10)** | 0.5 / 2 | 2Gi / 4Gi | none |
| titan-22 | `wger` (added live, see §4a) | **scavenger (-10)** | 0.25 / 1.2 | 0.6Gi / 2.25Gi | none |
| titan-22 | `sonarqube` (added live, see §4a) | **scavenger (-10)** | 0.1 / 2 | 2Gi / 4Gi | none |
| titan-22 | DaemonSets (dcgm-exporter, node-exporter, nvidia-process-exporter, fluent-bit, metis-sentinel-amd64, node-nofile, node-image-sweeper, titan-22-link-keeper) | default | ~0.3 total | ~0.9Gi total | none |
`jellyfin`'s live pod spec already carries `priorityClassName: media-core`,
@ -228,24 +229,28 @@ even though Brad has since made the call):
### 4a. Live update: Brad already implemented Option D (and Option B) directly
Between this audit's first pass and this revision, three direct commits to
`main` — **not through this card's PR, authored `jenkins <brad@bstein.dev>`**
— re-pinned Jenkins, VictoriaMetrics, and Wger onto titan-22:
Between this audit's first pass and this revision, **four** direct commits
to `main` — **not through this card's PR, authored
`jenkins <brad@bstein.dev>`** — re-pinned Jenkins, VictoriaMetrics, Wger,
and SonarQube onto titan-22, in rapid succession over about an hour:
| Commit | Workload | What changed |
|---|---|---|
| `271f3e8c3` "ops: spread saturated node workloads" | `jenkins` | `nodeSelector.arch` rpi5→amd64; required affinity `NotIn[titan-13,15,17,19]`+`hardware=rpi5` → required `hostname In [titan-22]`; added `priorityClassName: scavenger` and a `PreferNoSchedule` toleration for the media-primary taint |
| `271f3e8c3` (same commit) | `victoria-metrics-single-server` | required affinity `hostname NotIn [titan-12,20,21,22,24,...]` → `In [titan-22]`; added `priorityClassName: scavenger` + media-primary toleration; `upgrade.force: true` |
| `0ebab9d41` "placement(wger): use preemptible titan-22 capacity" | `wger` | same pattern: `NotIn[titan-13,15,17,19]` → `hostname In [titan-22]`, `arch` arm64→amd64, `priorityClassName: scavenger`, media-primary toleration |
| `17c5f5093` "placement(quality): use titan-22 spare capacity" | `sonarqube` | dropped its rpi5/rpi4 + control-plane-exclusion affinity and keycloak/grafana anti-affinity entirely; required affinity → `hostname In [titan-22]`, `arch` → amd64, `priorityClassName: scavenger`, media-primary toleration |
This is **exactly the Option B pattern** this document independently arrived
at (preemptible `scavenger` priority, CPU/RAM only, zero GPU requests) —
applied to Jenkins as well, which the first pass had flagged as needing a
separate evidence-based decision (Option D). Brad made that call directly.
None of these three request `nvidia.com/gpu*` (confirmed live). This is a
None of these four request `nvidia.com/gpu*` (confirmed live). This is a
direct-to-`main` change outside the "all durable changes via reviewable
titan-iac PRs" policy in this card's own body — noted here for the record,
not something this card can undo or should second-guess; it's the owner's
titan-iac PRs" policy in this card's own body, arriving faster than this
audit's own revision cycle (a 4th workload landed between this section
being drafted and being finalized) — noted here for the record, not
something this card can undo or should second-guess; it's the owner's
prerogative.
**Fresh evidence (14:40-14:45Z revalidation pass, ~1-3h of run time for the
@ -278,6 +283,11 @@ new placements) shows it is safe so far:**
titan-20 is now down to 1.7% memory available at its 3h floor (vs. "227
MiB / ~1.5%" at its 24h floor in the first pass) — Option A (leave
titan-20/21 alone) remains the only safe call for those two nodes.
- A 4th workload, `sonarqube` (`17c5f5093`), landed on titan-22 too late for
the 3h window above (~4 minutes of runtime at revalidation time) — too
little data for a trend, but it is `Running` with 0 restarts, requests
0.1 CPU / 2Gi / limits 2 CPU / 4Gi, `scavenger` priority, no GPU request,
consistent with the other three.
**New risk this live change introduces, not present in the first pass's
Option B analysis:** `victoria-metrics-single-server` is the cluster's