98 lines
4.5 KiB
Markdown
98 lines
4.5 KiB
Markdown
|
# Atlas Terraform
|
||
|
|
|
||
|
|
Terraform owns the declarative day-zero pieces that are not Flux workload state:
|
||
|
|
|
||
|
|
- Atlas host/node inventory used to generate Ananke config input.
|
||
|
|
- Required Kubernetes Node labels through `kubernetes_labels`.
|
||
|
|
- Optional render-only Flux day-zero manifests from `clusters/atlas/flux-system`.
|
||
|
|
|
||
|
|
Terraform does not own Flux-managed workload manifests. Do not move `services/`, `infrastructure/`, or Flux `Kustomization` trees under Terraform. Do not make manual `kubectl` edits for durable Kubernetes state.
|
||
|
|
|
||
|
|
## Ownership
|
||
|
|
|
||
|
|
Terraform owns:
|
||
|
|
|
||
|
|
- node names, IPs, roles, managed-node membership, ignored/unavailable markers
|
||
|
|
- default SSH user, port, config path, identity path emitted into Ananke inventory
|
||
|
|
- `startup.required_node_labels` values, applied as Kubernetes Node labels
|
||
|
|
- generated `ananke.inventory.yaml`
|
||
|
|
|
||
|
|
Flux owns:
|
||
|
|
|
||
|
|
- workloads, platform services, Helm releases, Flux `Kustomization` resources, and service manifests
|
||
|
|
- steady-state reconciliation of objects under `clusters/atlas/flux-system`, `infrastructure/`, and `services/`
|
||
|
|
|
||
|
|
Ananke owns:
|
||
|
|
|
||
|
|
- UPS shutdown decisions
|
||
|
|
- SSH repair and host recovery
|
||
|
|
- k3s restart/reboot
|
||
|
|
- Longhorn/runtime recovery
|
||
|
|
- cordon/uncordon, pod recycling, Flux suspend/resume during recovery
|
||
|
|
- startup validation/reporting for Terraform-owned labels when `required_node_labels_mode: validate`
|
||
|
|
|
||
|
|
## Run
|
||
|
|
|
||
|
|
```bash
|
||
|
|
cd terraform/atlas
|
||
|
|
cp terraform.tfvars.example terraform.tfvars
|
||
|
|
terraform init
|
||
|
|
terraform plan
|
||
|
|
```
|
||
|
|
|
||
|
|
For local syntax/provider validation that does not touch the cluster:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
terraform init -backend=false
|
||
|
|
terraform validate
|
||
|
|
```
|
||
|
|
|
||
|
|
Only run `terraform apply` after reviewing the plan and confirming label ownership. The Kubernetes provider uses `var.kubeconfig_path`, defaulting to `~/.kube/config`; no kubeconfig or secret material belongs in this repo.
|
||
|
|
|
||
|
|
The backend is local by default. When remote state is ready, migrate state with `terraform init -migrate-state` after adding the remote backend configuration in a separate reviewed change.
|
||
|
|
|
||
|
|
## Generated Ananke Fragment
|
||
|
|
|
||
|
|
The inventory module writes `generated/ananke.inventory.yaml` by default. It contains no secrets and is intended to be installed beside the main Ananke config:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
sudo ANANKE_INVENTORY_FRAGMENT=/opt/titan-iac/terraform/atlas/generated/ananke.inventory.yaml /opt/ananke/scripts/install.sh
|
||
|
|
```
|
||
|
|
|
||
|
|
Ananke auto-loads `/etc/ananke/ananke.inventory.yaml` when present. The generated fragment sets:
|
||
|
|
|
||
|
|
- `ssh_node_hosts`
|
||
|
|
- `ssh_managed_nodes`
|
||
|
|
- `control_planes`
|
||
|
|
- `workers`
|
||
|
|
- `startup.required_node_labels`
|
||
|
|
- `startup.required_node_labels_mode: validate`
|
||
|
|
- `startup.ignore_unavailable_nodes`
|
||
|
|
|
||
|
|
## Node Labels
|
||
|
|
|
||
|
|
This tree uses the HashiCorp Kubernetes provider `kubernetes_labels` resource. That resource manages only labels declared in Terraform and ignores existing labels not declared here. Leave `force_node_label_conflicts = false` for normal plans. Set it true only during a deliberate adoption when field ownership conflicts are understood.
|
||
|
|
|
||
|
|
## Flux Day-Zero
|
||
|
|
|
||
|
|
`clusters/atlas/flux-system/gotk-components.yaml` and `gotk-sync.yaml` already exist in the Flux bootstrap tree, so Terraform does not manage those objects by default.
|
||
|
|
|
||
|
|
The `flux_day_zero` module can render those artifacts into one file when `render_flux_day_zero_manifests = true`. Keep `apply_flux_day_zero_manifests = false` for steady state. Enabling apply runs `kubectl apply --server-side` and is only for an empty-cluster bootstrap with explicit operator intent.
|
||
|
|
|
||
|
|
## Migration Plan
|
||
|
|
|
||
|
|
1. Copy `terraform.tfvars.example` to `terraform.tfvars` and adjust only local paths.
|
||
|
|
2. Run `terraform init -backend=false` and `terraform validate`.
|
||
|
|
3. Run `terraform plan` in plan-only mode and review Node label changes.
|
||
|
|
4. Adopt existing label state with `kubernetes_labels` resources. This resource does not use `terraform import`; creating it against existing Nodes is the adoption step.
|
||
|
|
5. Install the generated Ananke fragment and confirm Ananke reads `required_node_labels_mode: validate`.
|
||
|
|
6. Run `terraform apply` for labels after the plan is clean.
|
||
|
|
7. Run Ananke startup dry-run/status and confirm it reports label drift instead of applying labels.
|
||
|
|
8. Enable in production after Ananke is validating only and Terraform is the only steady-state writer for these labels.
|
||
|
|
|
||
|
|
## Conflict Rules
|
||
|
|
|
||
|
|
- Do not add Terraform resources for any object already in Flux unless it is a one-time day-zero bootstrap action and Terraform will not keep steady-state ownership.
|
||
|
|
- Do not add Flux manifests for Node labels Terraform manages here.
|
||
|
|
- Do not use Terraform for recovery behavior that Ananke performs imperatively.
|