[hermes] ananke #295: known_hosts_repair_test_failure #14

New Issue

hermes-automation · 2026-08-29T18:20:18Z

hermes-automation commented

2026-08-29 18:20:18 +00:00

Hermes auto-triage classified incident ananke/295 as known_hosts_repair_test_failure (confidence 0.93); first failed gate: tests: TestRepairKnownHostsRemovesEntries.

Why a human is needed

The build first failed because the known-hosts repair behavior left a titan-0a SSH host-key entry in place when the test expected all titan-0a entries to be removed. Fix the implementation behind the repair path so its host matching and file rewrite logic consistently removes the targeted titan-0a records while preserving unrelated records. A separate vault lifecycle branch-matrix test later exceeded its ten-minute timeout and will still need follow-up once the first test failure is corrected.

Ariadne did not authorize automated remediation: classification_not_supported: 'known_hosts_repair_test_failure'.

Facts

jenkins — Jenkins build ananke #295 finished with result FAILURE after running for 1267.07 seconds. (https://ci.bstein.dev/job/ananke/295/)
jenkins — The build reported its first failed pipeline stage as "Enforce quality gate". (jenkins.first_failed_stage)
jenkins — The structured jenkins.failed_tests field was empty, so no published structured test result is available in the bundle. (jenkins.failed_tests)
jenkins — The earliest console failure excerpt records TestRepairKnownHostsRemovesEntries failing in internal/sshutil/repair_test.go:56 because a titan-0a known_hosts entry remained after repair. (jenkins.console_failures[1], line 222)
jenkins — The same console excerpt later records TestHookVaultLifecycleBranchMatrix timing out after 10 minutes. (jenkins.console_failures[1], line 222)
jenkins — The final quality-gate shell step exited with status 1, and Jenkins reported "script returned exit code 1". (jenkins.console_tail)
jenkins — The build checked out commit 76876ef895 with message "support terraform-owned bootstrap inputs". (jenkins.console_failures[1], line 222)
opensearch — OpenSearch correlation evidence was unavailable because the request failed with connection refused. (log_evidence.error)

Evidence

The earliest failure region of the build console:

tty: true
    volumeMounts:
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
  - command:
    - "cat"
    image: "registry.bstein.dev/bstein/quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64"
    name: "quality-tools"
    tty: true
    volumeMounts:
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
  - env:
    - name: "JENKINS_SECRET"
      value: "********"
    - name: "JENKINS_TUNNEL"
      value: "jenkins.jenkins.svc.cluster.local:50000"
    - name: "JENKINS_AGENT_NAME"
      value: "ananke-295-7m5rq-h4d9r-41r0g"

Inferences

The first enforced failure is the tests gate: TestRepairKnownHostsRemovesEntries is the first concrete failing test recorded chronologically. The later "Enforce quality gate" stage appears to aggregate that earlier test failure and terminate the pipeline, rather than being the original defect.
The known-hosts repair implementation did not remove the titan-0a entry expected by the test. The service maintainer should inspect the hostname matching and rewrite/removal logic used by the repair path, ensuring it removes the intended titan-0a entry without removing unrelated hosts such as tit...
The later vault lifecycle timeout is an additional failing test condition, but the retained console ordering does not make it the first enforced failure. It should be investigated after restoring the known-hosts repair test.
Sonar parser errors occur later in the console and the final shell variables show sonar_status=ok and supply_status=ok, so the supplied evidence does not show SonarQube or supply-chain policy as the cause of this build failure.

Links

Failed build: https://ci.bstein.dev/job/ananke/295/
Full evidence bundle and audit trail live in Ariadne at /api/admin/audit/events, event types hermes_autotriage_incident and hermes_autotriage_diagnosis.

Filed automatically by Ariadne from a Hermes Agent diagnosis (run run_5e920284b40b42b2a92a60538907190b). Hermes has no write access to this repository; no files or infrastructure were changed.

Hermes auto-triage classified incident `ananke/295` as **known_hosts_repair_test_failure** (confidence 0.93); first failed gate: `tests: TestRepairKnownHostsRemovesEntries`. ## Why a human is needed The build first failed because the known-hosts repair behavior left a titan-0a SSH host-key entry in place when the test expected all titan-0a entries to be removed. Fix the implementation behind the repair path so its host matching and file rewrite logic consistently removes the targeted titan-0a records while preserving unrelated records. A separate vault lifecycle branch-matrix test later exceeded its ten-minute timeout and will still need follow-up once the first test failure is corrected. Ariadne did not authorize automated remediation: `classification_not_supported: 'known_hosts_repair_test_failure'`. ## Facts - **jenkins** — Jenkins build ananke #295 finished with result FAILURE after running for 1267.07 seconds. (`https://ci.bstein.dev/job/ananke/295/`) - **jenkins** — The build reported its first failed pipeline stage as "Enforce quality gate". (`jenkins.first_failed_stage`) - **jenkins** — The structured jenkins.failed_tests field was empty, so no published structured test result is available in the bundle. (`jenkins.failed_tests`) - **jenkins** — The earliest console failure excerpt records TestRepairKnownHostsRemovesEntries failing in internal/sshutil/repair_test.go:56 because a titan-0a known_hosts entry remained after repair. (`jenkins.console_failures[1], line 222`) - **jenkins** — The same console excerpt later records TestHookVaultLifecycleBranchMatrix timing out after 10 minutes. (`jenkins.console_failures[1], line 222`) - **jenkins** — The final quality-gate shell step exited with status 1, and Jenkins reported "script returned exit code 1". (`jenkins.console_tail`) - **jenkins** — The build checked out commit 76876ef895bf48a8bf975cead2f4f7a962056373 with message "support terraform-owned bootstrap inputs". (`jenkins.console_failures[1], line 222`) - **opensearch** — OpenSearch correlation evidence was unavailable because the request failed with connection refused. (`log_evidence.error`) ## Evidence The earliest failure region of the build console: ``` tty: true volumeMounts: - mountPath: "/home/jenkins/agent" name: "workspace-volume" - command: - "cat" image: "registry.bstein.dev/bstein/quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64" name: "quality-tools" tty: true volumeMounts: - mountPath: "/home/jenkins/agent" name: "workspace-volume" - env: - name: "JENKINS_SECRET" value: "********" - name: "JENKINS_TUNNEL" value: "jenkins.jenkins.svc.cluster.local:50000" - name: "JENKINS_AGENT_NAME" value: "ananke-295-7m5rq-h4d9r-41r0g" ``` ## Inferences - The first enforced failure is the tests gate: TestRepairKnownHostsRemovesEntries is the first concrete failing test recorded chronologically. The later "Enforce quality gate" stage appears to aggregate that earlier test failure and terminate the pipeline, rather than being the original defect. - The known-hosts repair implementation did not remove the titan-0a entry expected by the test. The service maintainer should inspect the hostname matching and rewrite/removal logic used by the repair path, ensuring it removes the intended titan-0a entry without removing unrelated hosts such as tit... - The later vault lifecycle timeout is an additional failing test condition, but the retained console ordering does not make it the first enforced failure. It should be investigated after restoring the known-hosts repair test. - Sonar parser errors occur later in the console and the final shell variables show sonar_status=ok and supply_status=ok, so the supplied evidence does not show SonarQube or supply-chain policy as the cause of this build failure. ## Links - Failed build: https://ci.bstein.dev/job/ananke/295/ - Full evidence bundle and audit trail live in Ariadne at `/api/admin/audit/events`, event types `hermes_autotriage_incident` and `hermes_autotriage_diagnosis`. Filed automatically by Ariadne from a Hermes Agent diagnosis (run [run_5e920284b40b42b2a92a60538907190b](https://triage.bstein.dev/chat?resume=run_5e920284b40b42b2a92a60538907190b)). Hermes has no write access to this repository; no files or infrastructure were changed.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: titan/ananke#14