[hermes] soteria #297: sonarqube_quality_gate_failure #10

New Issue

bstein · 2026-08-15T21:27:32Z

bstein commented

2026-08-15 21:27:32 +00:00

Hermes auto-triage classified incident soteria/297 as sonarqube_quality_gate_failure (confidence 0.55); first failed gate: Enforce quality gate.

Why a human is needed

Build 297 got through building and running the SonarQube scan but was rejected at the 'Enforce quality gate' stage, meaning the analysis was submitted and SonarQube's quality gate check on it did not pass. While the scan was running, the Go analyzer failed to parse 20 of 26 Go source files (errors like 'Go parser external process returned non-zero exit value: 2', with two files failing on a broken pipe instead), and the TypeScript/JavaScript sensor couldn't even start because the analysis container has no 'node' binary. Neither of these stopped the scan outright — it still reported all 26 Go files as 'analyzed' and moved on through the remaining sensors — so it isn't certain they're what tripped the gate. The retained console log is truncated and doesn't include the part where Jenkins reports which specific quality gate condition (coverage, new issues, duplications, etc.) actually failed, so the precise reason the gate rejected this build can't be confirmed from this evidence alone. To pin it down, check the SonarQube project's quality gate history/dashboard for this analysis, or pull the untruncated console log around the 'Enforce quality gate' stage. Separately, worth fixing regardless of the gate outcome: the quality-tools image used for this build (registry.bstein.dev/bstein/quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64) is missing a Node.js binary needed for JS/TS analysis, and the Go parser is failing on the majority of the repo's Go files — both point to a broken or mismatched analysis toolchain that is degrading analysis coverage on every build using this image, independent of whatever caused this specific gate failure.

Ariadne did not authorize automated remediation: human_required.

Facts

jenkins — Jenkins build soteria/297 ended with result FAILURE; the pipeline's first failed stage was 'Enforce quality gate'. (first_failed_stage="Enforce quality gate", result="FAILURE")
jenkins — No structured test results were published for this build. (failed_tests=[])
jenkins — During the SonarQube scan, the Go analyzer failed to parse 20 of 26 Go source files, e.g. "Cannot parse 'internal/k8s/state.go': Go parser external process returned non-zero exit value: 2", with two files (internal/server/server.go, internal/server/server_utilities.go) failing on 'Broken pipe' in... (console_failures[1] (marker "non-zero exit"), line_number 291, 21:03:04–21:03:05)
jenkins — The TypeScript/JavaScript sensor could not start because no Node.js binary is present in the analysis container: "Error when running: 'node -v'. Is Node.js available during analysis?" / "Cannot run program 'node': error=2, No such file or directory"; the sensor nonetheless reported 0 JS/TS files ... (console_failures[1]-[2] (markers "non-zero exit", "No such file"), line_number 291-394, 21:03:28)
jenkins — Despite the Go parse errors, the scan logged all 26 Go source files as analyzed and continued through the remaining sensors (JaCoCo, IaC CloudFormation, IaC Kubernetes) before the pipeline terminated; the console tail only shows generic post-failure teardown (Build & push image skipped, script ex... (console_failures[1] and console_tail)
jenkins — The retained console excerpt for this build is marked truncated, and no records were returned from the correlated log query for the build's time window. (console_truncated=true)
opensearch — No supporting records were found in the log_evidence source for this build's time window. (log_evidence.records=[], query_window 2026-08-15T20:54:00Z–21:28:21Z)

Evidence

The earliest failure region of the build console:

tty: true
    volumeMounts:
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
  - command:
    - "cat"
    image: "registry.bstein.dev/bstein/quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64"
    name: "quality-tools"
    tty: true
    volumeMounts:
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
  - env:
    - name: "JENKINS_SECRET"
      value: "********"
    - name: "JENKINS_TUNNEL"
      value: "jenkins.jenkins.svc.cluster.local:50000"
    - name: "JENKINS_AGENT_NAME"
      value: "soteria-297-pj54x-gqdl5-1qgv1"

Inferences

The Go parser failures on 20 of 26 files and the missing Node.js binary degraded SonarQube's analysis coverage for this build, but the scan still completed ('26/26 source files have been analyzed') and continued sensor-by-sensor, so these errors are not confirmed as the direct cause of the qualit...
The specific quality gate condition that actually failed (coverage, new bugs, duplications, etc.) is not visible in the retained console excerpts; the console is marked truncated and the section immediately around the 'Enforce quality gate' stage result is not included in this bundle.
The Node.js error occurred with 0 JavaScript/TypeScript files reported for analysis, so on its own it likely had no practical effect on this build's quality-gate outcome even though it shows the analysis image is missing a capability it expects to have.

Suggested remediation (not available)

No automated remediation exists for this failure. Hermes proposes one; it was not performed, and Ariadne cannot perform it until a person builds and deploys it.

Proposed action: capture_sonarqube_gate_conditions
What it would do: When the 'Enforce quality gate' stage fails, automatically call the SonarQube API for the analysis's quality gate detail (the failed condition(s) and their measured values) and append it to the Jenkins console output, rather than letting the pipeline surface only the generic exit-code failure.
Evidence that should be required first: A build's 'Enforce quality gate' stage log lacking any condition-level breakdown from SonarQube (as seen in this incident), confirming the pipeline currently does not surface which specific metric caused the gate to fail.

Links

Failed build: https://ci.bstein.dev/job/soteria/297/
Full evidence bundle and audit trail live in Ariadne at /api/admin/audit/events, event types hermes_autotriage_incident and hermes_autotriage_diagnosis.

Filed automatically by Ariadne from a Hermes Agent diagnosis (run run_0bfcb6d5a36b47558f75cd4f09f1837f). Hermes has no write access to this repository; no files or infrastructure were changed.

Hermes auto-triage classified incident `soteria/297` as **sonarqube_quality_gate_failure** (confidence 0.55); first failed gate: `Enforce quality gate`. ## Why a human is needed Build 297 got through building and running the SonarQube scan but was rejected at the 'Enforce quality gate' stage, meaning the analysis was submitted and SonarQube's quality gate check on it did not pass. While the scan was running, the Go analyzer failed to parse 20 of 26 Go source files (errors like 'Go parser external process returned non-zero exit value: 2', with two files failing on a broken pipe instead), and the TypeScript/JavaScript sensor couldn't even start because the analysis container has no 'node' binary. Neither of these stopped the scan outright — it still reported all 26 Go files as 'analyzed' and moved on through the remaining sensors — so it isn't certain they're what tripped the gate. The retained console log is truncated and doesn't include the part where Jenkins reports which specific quality gate condition (coverage, new issues, duplications, etc.) actually failed, so the precise reason the gate rejected this build can't be confirmed from this evidence alone. To pin it down, check the SonarQube project's quality gate history/dashboard for this analysis, or pull the untruncated console log around the 'Enforce quality gate' stage. Separately, worth fixing regardless of the gate outcome: the quality-tools image used for this build (registry.bstein.dev/bstein/quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64) is missing a Node.js binary needed for JS/TS analysis, and the Go parser is failing on the majority of the repo's Go files — both point to a broken or mismatched analysis toolchain that is degrading analysis coverage on every build using this image, independent of whatever caused this specific gate failure. Ariadne did not authorize automated remediation: `human_required`. ## Facts - **jenkins** — Jenkins build soteria/297 ended with result FAILURE; the pipeline's first failed stage was 'Enforce quality gate'. (`first_failed_stage="Enforce quality gate", result="FAILURE"`) - **jenkins** — No structured test results were published for this build. (`failed_tests=[]`) - **jenkins** — During the SonarQube scan, the Go analyzer failed to parse 20 of 26 Go source files, e.g. "Cannot parse 'internal/k8s/state.go': Go parser external process returned non-zero exit value: 2", with two files (internal/server/server.go, internal/server/server_utilities.go) failing on 'Broken pipe' in... (`console_failures[1] (marker "non-zero exit"), line_number 291, 21:03:04–21:03:05`) - **jenkins** — The TypeScript/JavaScript sensor could not start because no Node.js binary is present in the analysis container: "Error when running: 'node -v'. Is Node.js available during analysis?" / "Cannot run program 'node': error=2, No such file or directory"; the sensor nonetheless reported 0 JS/TS files ... (`console_failures[1]-[2] (markers "non-zero exit", "No such file"), line_number 291-394, 21:03:28`) - **jenkins** — Despite the Go parse errors, the scan logged all 26 Go source files as analyzed and continued through the remaining sensors (JaCoCo, IaC CloudFormation, IaC Kubernetes) before the pipeline terminated; the console tail only shows generic post-failure teardown (Build & push image skipped, script ex... (`console_failures[1] and console_tail`) - **jenkins** — The retained console excerpt for this build is marked truncated, and no records were returned from the correlated log query for the build's time window. (`console_truncated=true`) - **opensearch** — No supporting records were found in the log_evidence source for this build's time window. (`log_evidence.records=[], query_window 2026-08-15T20:54:00Z–21:28:21Z`) ## Evidence The earliest failure region of the build console: ``` tty: true volumeMounts: - mountPath: "/home/jenkins/agent" name: "workspace-volume" - command: - "cat" image: "registry.bstein.dev/bstein/quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64" name: "quality-tools" tty: true volumeMounts: - mountPath: "/home/jenkins/agent" name: "workspace-volume" - env: - name: "JENKINS_SECRET" value: "********" - name: "JENKINS_TUNNEL" value: "jenkins.jenkins.svc.cluster.local:50000" - name: "JENKINS_AGENT_NAME" value: "soteria-297-pj54x-gqdl5-1qgv1" ``` ## Inferences - The Go parser failures on 20 of 26 files and the missing Node.js binary degraded SonarQube's analysis coverage for this build, but the scan still completed ('26/26 source files have been analyzed') and continued sensor-by-sensor, so these errors are not confirmed as the direct cause of the qualit... - The specific quality gate condition that actually failed (coverage, new bugs, duplications, etc.) is not visible in the retained console excerpts; the console is marked truncated and the section immediately around the 'Enforce quality gate' stage result is not included in this bundle. - The Node.js error occurred with 0 JavaScript/TypeScript files reported for analysis, so on its own it likely had no practical effect on this build's quality-gate outcome even though it shows the analysis image is missing a capability it expects to have. ## Suggested remediation (not available) No automated remediation exists for this failure. Hermes proposes one; it was not performed, and Ariadne cannot perform it until a person builds and deploys it. - Proposed action: `capture_sonarqube_gate_conditions` - What it would do: When the 'Enforce quality gate' stage fails, automatically call the SonarQube API for the analysis's quality gate detail (the failed condition(s) and their measured values) and append it to the Jenkins console output, rather than letting the pipeline surface only the generic exit-code failure. - Evidence that should be required first: A build's 'Enforce quality gate' stage log lacking any condition-level breakdown from SonarQube (as seen in this incident), confirming the pipeline currently does not surface which specific metric caused the gate to fail. ## Links - Failed build: https://ci.bstein.dev/job/soteria/297/ - Full evidence bundle and audit trail live in Ariadne at `/api/admin/audit/events`, event types `hermes_autotriage_incident` and `hermes_autotriage_diagnosis`. Filed automatically by Ariadne from a Hermes Agent diagnosis (run [run_0bfcb6d5a36b47558f75cd4f09f1837f](https://triage.hermes.bstein.dev/chat?resume=run_0bfcb6d5a36b47558f75cd4f09f1837f)). Hermes has no write access to this repository; no files or infrastructure were changed.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: titan/soteria#10