titan/soteria
3
0
Fork 0
Code Issues 23 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

[hermes] soteria #348: sonarqube_analyzer_runtime_missing #23

New Issue
Open
opened 2026-09-10 09:31:15 +00:00 by hermes-automation · 0 comments
hermes-automation commented 2026-09-10 09:31:15 +00:00
Member
Copy Link

Hermes auto-triage classified incident soteria/348 as sonarqube_analyzer_runtime_missing (confidence 0.6); first failed gate: sonarqube.

Why a human is needed

The build failed in the 'Enforce quality gate' stage. The only errors recorded anywhere in the retained log come from the SonarQube scanner at 09:06: every one of the 26 Go files failed to parse with process-level errors from the analyzer's external Go parser ('returned non-zero exit value: 2', 'Broken pipe', 'Stream closed'), and the JavaScript/TypeScript analyzer aborted because there is no 'node' executable in the quality-tools container ('Cannot run program "node": error=2, No such file or directory'). These are toolchain problems inside the analysis image, not defects in soteria's code or tests; no test results were published and nothing in the log points at a failing assertion. With no Go file successfully parsed, the analysis reports essentially no code for the project, which is enough to make a metric-based gate fail. The fix is to rebuild the arm64 quality-tools image with a Node.js runtime and a working Go analyzer, confirm 'node -v' and a scanner run succeed inside it, and re-run the commit. A maintainer is needed because the console is truncated between the analysis at 09:06 and the build ending at 09:24, so the line that actually returned exit code 1 is not retained; the full console at https://ci.bstein.dev/job/soteria/348/consoleFull should be checked to rule out a second, unrelated gate failure in that window. Centralised logs were also unavailable for the incident window (connection refused), so Jenkins output is the only evidence available.

Ariadne did not authorize automated remediation: human_required.

Facts

  • jenkins — Build soteria #348 ran 2026-09-10T08:59:00Z to 2026-09-10T09:24:43Z (1543s) and ended with result FAILURE. (https://ci.bstein.dev/job/soteria/348/ (jenkins.result, jenkins.timestamps))
  • jenkins — The first pipeline stage Jenkins reported as failed is 'Enforce quality gate'; the later 'Build & push image' stage was skipped due to earlier failure(s). (jenkins.first_failed_stage; jenkins.console_tail)
  • jenkins — No structured test results were published for this build: jenkins.failed_tests is an empty array. (jenkins.failed_tests)
  • jenkins — The SonarQube analysis ran inside the container image registry.bstein.dev/bstein/quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64 in the build pod. (jenkins.console_failures[0], console line 139 (pod template))
  • jenkins — Between 09:06:17 and 09:06:18 the Sonar Go sensor failed to parse all 26 Go source files under /home/jenkins/agent/workspace/Soteria, reporting 'Go parser external process returned non-zero exit value: 2' for most files and 'Broken pipe' / 'Stream closed' for internal/k8s/state.go, internal/k8s/j... (jenkins.console_failures[1], console line ~334)
  • jenkins — At 09:06:33 the TypeScript/JavaScript sensor aborted with org.sonarsource.nodejs.NodeCommandException: "Error when running: 'node -v'. Is Node.js available during analysis?" (jenkins.console_failures[1], console line ~334)
  • jenkins — The underlying cause of that exception is java.io.IOException: Cannot run program "node": error=2, No such file or directory. (jenkins.console_failures[2], console line 435)
  • jenkins — The build ended with 'ERROR: script returned exit code 1' at the end of the pipeline. (jenkins.console_tail; jenkins.console_failures[3], console line 3381)
  • jenkins — The console log is truncated (console_truncated=true) and no failure excerpt was retained between line 435 (09:06:33) and line 3381 (end of build), leaving roughly 18 minutes of the run without retained detail. (jenkins.console_truncated; jenkins.console_failures ordering)
  • opensearch — No centralised log records could be retrieved for the incident window: the log query returned 'opensearch request failed: [Errno 111] Connection refused' with zero records. (log_evidence.error; log_evidence.query_window 08:54:00Z-09:29:43Z)

Evidence

The earliest failure region of the build console:

tty: true
    volumeMounts:
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
  - command:
    - "cat"
    image: "registry.bstein.dev/bstein/quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64"
    name: "quality-tools"
    tty: true
    volumeMounts:
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
  - env:
    - name: "JENKINS_SECRET"
      value: "********"
    - name: "JENKINS_TUNNEL"
      value: "jenkins.jenkins.svc.cluster.local:50000"
    - name: "JENKINS_AGENT_NAME"
      value: "soteria-348-kcjk6-ckn7p-1s098"

Inferences

  • The static-analysis step could not actually analyse this repository. The container image used for analysis (quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64) has no 'node' binary on PATH, so SonarQube's JavaScript/TypeScript analyzer could not start at all; that is what the 'Cannot run progr...
  • The Go analysis failed for the same environmental reason rather than because of anything in the source. All 26 Go files failed, and the errors are process-level ('external process returned non-zero exit value: 2', 'Broken pipe', 'Stream closed') rather than per-file syntax diagnostics. A genuine ...
  • Because no Go file was parsed, SonarQube received effectively zero analysable code for the project, so any gate condition computed from Sonar metrics (coverage, issues, duplication on new code) is evaluated against empty data. That is the most plausible chain from the analyzer errors to the quali...
  • A fix is on the CI toolchain side, not in soteria's source. The quality-tools image needs to be rebuilt for arm64 with a Node.js runtime installed and a Go analyzer that actually runs (verify inside the container with 'node -v' and by running the scanner against a single Go file), then the same c...
  • Important caveat: the console log for this build is truncated between the analysis at 09:06 and the end at 09:24, so the exact line that produced 'exit code 1' is not in the retained evidence. The analyzer errors above are the only recorded errors in the build, but the full console at https://ci....

Suggested remediation (not available)

No automated remediation exists for this failure. Hermes proposes one; it was not performed, and Ariadne cannot perform it until a person builds and deploys it.

  • Proposed action: republish_quality_tools_analysis_image
  • What it would do: Rebuild and republish the registry.bstein.dev/bstein/quality-tools image for arm64 so it contains a working Node.js runtime and a functioning Go analyzer binary, verify both inside the image, then re-run the failing build on the same commit.
  • Evidence that should be required first: The build console must show the Sonar scanner failing to launch an analyzer runtime inside the quality-tools container (for example 'Cannot run program "node"' or the Go parser subprocess exiting non-zero for every file), the analysis must be confirmed to have run in that image, and there must be no repository commit in the build's range that plausibly explains the parse failures.

Links

  • Failed build: https://ci.bstein.dev/job/soteria/348/
  • Full evidence bundle and audit trail live in Ariadne at /api/admin/audit/events, event types hermes_autotriage_incident and hermes_autotriage_diagnosis.

Filed automatically by Ariadne from a Hermes Agent diagnosis (run run_27639d134c894583ae525a8965d55ab2). Hermes has no write access to this repository; no files or infrastructure were changed.

Hermes auto-triage classified incident `soteria/348` as **sonarqube_analyzer_runtime_missing** (confidence 0.6); first failed gate: `sonarqube`. ## Why a human is needed The build failed in the 'Enforce quality gate' stage. The only errors recorded anywhere in the retained log come from the SonarQube scanner at 09:06: every one of the 26 Go files failed to parse with process-level errors from the analyzer's external Go parser ('returned non-zero exit value: 2', 'Broken pipe', 'Stream closed'), and the JavaScript/TypeScript analyzer aborted because there is no 'node' executable in the quality-tools container ('Cannot run program "node": error=2, No such file or directory'). These are toolchain problems inside the analysis image, not defects in soteria's code or tests; no test results were published and nothing in the log points at a failing assertion. With no Go file successfully parsed, the analysis reports essentially no code for the project, which is enough to make a metric-based gate fail. The fix is to rebuild the arm64 quality-tools image with a Node.js runtime and a working Go analyzer, confirm 'node -v' and a scanner run succeed inside it, and re-run the commit. A maintainer is needed because the console is truncated between the analysis at 09:06 and the build ending at 09:24, so the line that actually returned exit code 1 is not retained; the full console at https://ci.bstein.dev/job/soteria/348/consoleFull should be checked to rule out a second, unrelated gate failure in that window. Centralised logs were also unavailable for the incident window (connection refused), so Jenkins output is the only evidence available. Ariadne did not authorize automated remediation: `human_required`. ## Facts - **jenkins** — Build soteria #348 ran 2026-09-10T08:59:00Z to 2026-09-10T09:24:43Z (1543s) and ended with result FAILURE. (`https://ci.bstein.dev/job/soteria/348/ (jenkins.result, jenkins.timestamps)`) - **jenkins** — The first pipeline stage Jenkins reported as failed is 'Enforce quality gate'; the later 'Build & push image' stage was skipped due to earlier failure(s). (`jenkins.first_failed_stage; jenkins.console_tail`) - **jenkins** — No structured test results were published for this build: jenkins.failed_tests is an empty array. (`jenkins.failed_tests`) - **jenkins** — The SonarQube analysis ran inside the container image registry.bstein.dev/bstein/quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64 in the build pod. (`jenkins.console_failures[0], console line 139 (pod template)`) - **jenkins** — Between 09:06:17 and 09:06:18 the Sonar Go sensor failed to parse all 26 Go source files under /home/jenkins/agent/workspace/Soteria, reporting 'Go parser external process returned non-zero exit value: 2' for most files and 'Broken pipe' / 'Stream closed' for internal/k8s/state.go, internal/k8s/j... (`jenkins.console_failures[1], console line ~334`) - **jenkins** — At 09:06:33 the TypeScript/JavaScript sensor aborted with org.sonarsource.nodejs.NodeCommandException: "Error when running: 'node -v'. Is Node.js available during analysis?" (`jenkins.console_failures[1], console line ~334`) - **jenkins** — The underlying cause of that exception is java.io.IOException: Cannot run program "node": error=2, No such file or directory. (`jenkins.console_failures[2], console line 435`) - **jenkins** — The build ended with 'ERROR: script returned exit code 1' at the end of the pipeline. (`jenkins.console_tail; jenkins.console_failures[3], console line 3381`) - **jenkins** — The console log is truncated (console_truncated=true) and no failure excerpt was retained between line 435 (09:06:33) and line 3381 (end of build), leaving roughly 18 minutes of the run without retained detail. (`jenkins.console_truncated; jenkins.console_failures ordering`) - **opensearch** — No centralised log records could be retrieved for the incident window: the log query returned 'opensearch request failed: [Errno 111] Connection refused' with zero records. (`log_evidence.error; log_evidence.query_window 08:54:00Z-09:29:43Z`) ## Evidence The earliest failure region of the build console: ``` tty: true volumeMounts: - mountPath: "/home/jenkins/agent" name: "workspace-volume" - command: - "cat" image: "registry.bstein.dev/bstein/quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64" name: "quality-tools" tty: true volumeMounts: - mountPath: "/home/jenkins/agent" name: "workspace-volume" - env: - name: "JENKINS_SECRET" value: "********" - name: "JENKINS_TUNNEL" value: "jenkins.jenkins.svc.cluster.local:50000" - name: "JENKINS_AGENT_NAME" value: "soteria-348-kcjk6-ckn7p-1s098" ``` ## Inferences - The static-analysis step could not actually analyse this repository. The container image used for analysis (quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64) has no 'node' binary on PATH, so SonarQube's JavaScript/TypeScript analyzer could not start at all; that is what the 'Cannot run progr... - The Go analysis failed for the same environmental reason rather than because of anything in the source. All 26 Go files failed, and the errors are process-level ('external process returned non-zero exit value: 2', 'Broken pipe', 'Stream closed') rather than per-file syntax diagnostics. A genuine ... - Because no Go file was parsed, SonarQube received effectively zero analysable code for the project, so any gate condition computed from Sonar metrics (coverage, issues, duplication on new code) is evaluated against empty data. That is the most plausible chain from the analyzer errors to the quali... - A fix is on the CI toolchain side, not in soteria's source. The quality-tools image needs to be rebuilt for arm64 with a Node.js runtime installed and a Go analyzer that actually runs (verify inside the container with 'node -v' and by running the scanner against a single Go file), then the same c... - Important caveat: the console log for this build is truncated between the analysis at 09:06 and the end at 09:24, so the exact line that produced 'exit code 1' is not in the retained evidence. The analyzer errors above are the only recorded errors in the build, but the full console at https://ci.... ## Suggested remediation (not available) No automated remediation exists for this failure. Hermes proposes one; it was not performed, and Ariadne cannot perform it until a person builds and deploys it. - Proposed action: `republish_quality_tools_analysis_image` - What it would do: Rebuild and republish the registry.bstein.dev/bstein/quality-tools image for arm64 so it contains a working Node.js runtime and a functioning Go analyzer binary, verify both inside the image, then re-run the failing build on the same commit. - Evidence that should be required first: The build console must show the Sonar scanner failing to launch an analyzer runtime inside the quality-tools container (for example 'Cannot run program "node"' or the Go parser subprocess exiting non-zero for every file), the analysis must be confirmed to have run in that image, and there must be no repository commit in the build's range that plausibly explains the parse failures. ## Links - Failed build: https://ci.bstein.dev/job/soteria/348/ - Full evidence bundle and audit trail live in Ariadne at `/api/admin/audit/events`, event types `hermes_autotriage_incident` and `hermes_autotriage_diagnosis`. Filed automatically by Ariadne from a Hermes Agent diagnosis (run [run_27639d134c894583ae525a8965d55ab2](https://triage.bstein.dev/chat?resume=run_27639d134c894583ae525a8965d55ab2)). Hermes has no write access to this repository; no files or infrastructure were changed. <!-- hermes-triage job=soteria classification=sonarqube_analyzer_runtime_missing incident=soteria/348 -->
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
atlas-reconciler bstein hermes-automation
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: titan/soteria#23
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?