titan/soteria
3
0
Fork 0
Code Issues 23 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

[hermes] soteria #330: sonarqube_scanner_toolchain_missing #18

New Issue
Open
opened 2026-09-01 09:25:28 +00:00 by hermes-automation · 0 comments
hermes-automation commented 2026-09-01 09:25:28 +00:00
Member
Copy Link

Hermes auto-triage classified incident soteria/330 as sonarqube_scanner_toolchain_missing (confidence 0.55); first failed gate: sonarqube.

Why a human is needed

Build 330 failed in the "Enforce quality gate" stage. No test results were published and no test failure is recorded, so this is not a code regression caught by the test suite. The earliest errors in the build come from the SonarQube scanner: between 09:02:49 and 09:02:50 every one of the 26 Go source files failed with "Go parser external process returned non-zero exit value: 2", and at 09:03:07 the JS/TS sensor aborted with "Cannot run program "node": error=2, No such file or directory". A uniform failure across all 26 files, main.go included, is not a source-code defect - the analyzer's external parser process cannot run inside the scanner container (missing or wrong-architecture binary; these agents are arm64), and Node.js is simply not present in that image. The scan consequently produced no Go measurements and imported no coverage report, which is consistent with the quality-gate check exiting 1 later in the build. Fixing this means changing the CI scanner image and pipeline, not Soteria's code: provide a Node.js runtime on PATH, provide a Go analyzer build that executes on the agent architecture (or pin the scan to a matching architecture), and make the scan step fail at the point an analyzer cannot start rather than logging errors and letting a later gate check fail with a bare exit code 1. A maintainer is needed because the exact command that returned exit code 1 sits in the truncated part of the console log and the scanner image contents must be inspected before it is rebuilt; application logs were unavailable for this window (OpenSearch connection refused), so there is no second source to corroborate the timeline.

Ariadne did not authorize automated remediation: human_required.

Facts

  • jenkins — Build soteria/330 finished with result FAILURE, running from 2026-09-01T08:59:00.292Z to 2026-09-01T09:21:57.097Z (1376.8s). (https://ci.bstein.dev/job/soteria/330/ (jenkins.result, jenkins.timestamps))
  • jenkins — Jenkins reports the failing pipeline stage as "Enforce quality gate"; the later "Build & push image" stage was skipped due to earlier failure(s). (jenkins.first_failed_stage; jenkins.console_tail)
  • jenkins — The build published no structured test results and records no failing tests. (jenkins.failed_tests = [])
  • jenkins — In the earliest console failure region, the SonarQube Go sensor failed to parse all 26 Go source files between 09:02:49.370 and 09:02:50.190, each with "Go parser external process returned non-zero exit value: 2" (e.g. internal/k8s/state.go, internal/config/config.go, cmd/soteria/main.go). (jenkins.console_failures[0], line 296)
  • jenkins — At 09:03:07.710 the TypeScript/JavaScript sensor failed with NodeCommandException: "Error when running: 'node -v'. Is Node.js available during analysis?" (jenkins.console_failures[0], line 296)
  • jenkins — The root cause of that sensor error is "java.io.IOException: Cannot run program "node": error=2, No such file or directory" - the node binary is absent from the scanner environment. (jenkins.console_failures[1], line 399)
  • jenkins — No coverage data was imported during the scan: "No report imported, no coverage information will be imported by JaCoCo XML Report Importer". (jenkins.console_failures[0], line 296)
  • jenkins — The build ends with "ERROR: script returned exit code 1" followed by "Finished: FAILURE", with no further diagnostic text. (jenkins.console_failures[2], line 3345; jenkins.console_tail)
  • jenkins — The console log is truncated in the retained evidence, so the roughly 2900 lines between the scanner errors (line 399) and the final exit-code-1 line (line 3345) are not available. (jenkins.console_truncated = true)
  • opensearch — No application log records could be retrieved for the incident window 08:54:00Z-09:26:57Z; the log query failed with "opensearch request failed: [Errno 111] Connection refused". (log_evidence.error)

Evidence

The earliest failure region of the build console:

09:02:50.252 INFO  Sensor JaCoCo XML Report Importer [jacoco] (done) | time=60ms
09:02:50.253 INFO  Sensor IaC CloudFormation Sensor [iac]
09:02:50.491 INFO  0 source files to be analyzed
09:02:50.642 INFO  0/0 source files have been analyzed
09:02:50.643 INFO  Sensor IaC CloudFormation Sensor [iac] (done) | time=390ms
09:02:50.644 INFO  Sensor IaC Kubernetes Sensor [iac]
09:02:50.870 INFO  2 source files to be analyzed
09:02:51.412 INFO  2/2 source files have been analyzed
09:02:51.450 INFO  Sensor IaC Kubernetes Sensor [iac] (done) | time=806ms
09:02:51.450 INFO  Sensor TypeScript analysis [javascript]
09:03:07.710 ERROR Error when running: 'node -v'. Is Node.js available during analysis?
org.sonarsource.nodejs.NodeCommandException: Error when running: 'node -v'. Is Node.js available during analysis?
	at org.sonarsource.nodejs.NodeCommand.start(NodeCommand.java:79)
	at org.sonarsource.nodejs.NodeCommandBuilderImpl.getVersion(NodeCommandBuilderImpl.java:203)
	at org.sonarsource.nodejs.NodeCommandBuilderImpl.checkNodeCompatibility(NodeCommandBuilderImpl.java:169)
	at org.sonarsource.nodejs.NodeCommandBuilderImpl.build(NodeCommandBuilderImpl.java:143)
	at org.sonar.plugins.javascrip
... (truncated; the full log is in the linked build)

Inferences

  • Nothing in Soteria's test suite failed. No test report was published and no failing test is recorded, and the stage that failed is the quality-gate stage, not a test stage.
  • The Go analyzer failure is environmental, not a source-code problem. All 26 Go files failed identically with exit status 2 from an external helper process, including cmd/soteria/main.go. A real syntax error would affect specific files, not every file in the repository, and the same tree compiles ...
  • Node.js is not installed in the scanner container image. "Cannot run program "node": error=2, No such file or directory" is a missing executable on PATH, not a network, DNS, or registry problem, so re-running the same commit on the same image will reproduce it exactly.
  • Because zero Go files were analyzed and no coverage report was imported, the analysis submitted for this commit contained effectively none of the repository's measurements. The subsequent quality-gate check therefore evaluated a project with empty or zero coverage/issue data, which is the most pl...
  • A fix belongs in the CI scanner image and pipeline rather than in Soteria's source: the scanner container needs a Node.js runtime on PATH and a Go analyzer that can execute on the agent's architecture (or the scan must run on a matching architecture). Separately, the scan step should fail immedia...

Links

  • Failed build: https://ci.bstein.dev/job/soteria/330/
  • Full evidence bundle and audit trail live in Ariadne at /api/admin/audit/events, event types hermes_autotriage_incident and hermes_autotriage_diagnosis.

Filed automatically by Ariadne from a Hermes Agent diagnosis (run run_5b7d8e8a6b3e4860902ba7e637fd98a8). Hermes has no write access to this repository; no files or infrastructure were changed.

Hermes auto-triage classified incident `soteria/330` as **sonarqube_scanner_toolchain_missing** (confidence 0.55); first failed gate: `sonarqube`. ## Why a human is needed Build 330 failed in the "Enforce quality gate" stage. No test results were published and no test failure is recorded, so this is not a code regression caught by the test suite. The earliest errors in the build come from the SonarQube scanner: between 09:02:49 and 09:02:50 every one of the 26 Go source files failed with "Go parser external process returned non-zero exit value: 2", and at 09:03:07 the JS/TS sensor aborted with "Cannot run program \"node\": error=2, No such file or directory". A uniform failure across all 26 files, main.go included, is not a source-code defect - the analyzer's external parser process cannot run inside the scanner container (missing or wrong-architecture binary; these agents are arm64), and Node.js is simply not present in that image. The scan consequently produced no Go measurements and imported no coverage report, which is consistent with the quality-gate check exiting 1 later in the build. Fixing this means changing the CI scanner image and pipeline, not Soteria's code: provide a Node.js runtime on PATH, provide a Go analyzer build that executes on the agent architecture (or pin the scan to a matching architecture), and make the scan step fail at the point an analyzer cannot start rather than logging errors and letting a later gate check fail with a bare exit code 1. A maintainer is needed because the exact command that returned exit code 1 sits in the truncated part of the console log and the scanner image contents must be inspected before it is rebuilt; application logs were unavailable for this window (OpenSearch connection refused), so there is no second source to corroborate the timeline. Ariadne did not authorize automated remediation: `human_required`. ## Facts - **jenkins** — Build soteria/330 finished with result FAILURE, running from 2026-09-01T08:59:00.292Z to 2026-09-01T09:21:57.097Z (1376.8s). (`https://ci.bstein.dev/job/soteria/330/ (jenkins.result, jenkins.timestamps)`) - **jenkins** — Jenkins reports the failing pipeline stage as "Enforce quality gate"; the later "Build & push image" stage was skipped due to earlier failure(s). (`jenkins.first_failed_stage; jenkins.console_tail`) - **jenkins** — The build published no structured test results and records no failing tests. (`jenkins.failed_tests = []`) - **jenkins** — In the earliest console failure region, the SonarQube Go sensor failed to parse all 26 Go source files between 09:02:49.370 and 09:02:50.190, each with "Go parser external process returned non-zero exit value: 2" (e.g. internal/k8s/state.go, internal/config/config.go, cmd/soteria/main.go). (`jenkins.console_failures[0], line 296`) - **jenkins** — At 09:03:07.710 the TypeScript/JavaScript sensor failed with NodeCommandException: "Error when running: 'node -v'. Is Node.js available during analysis?" (`jenkins.console_failures[0], line 296`) - **jenkins** — The root cause of that sensor error is "java.io.IOException: Cannot run program \"node\": error=2, No such file or directory" - the node binary is absent from the scanner environment. (`jenkins.console_failures[1], line 399`) - **jenkins** — No coverage data was imported during the scan: "No report imported, no coverage information will be imported by JaCoCo XML Report Importer". (`jenkins.console_failures[0], line 296`) - **jenkins** — The build ends with "ERROR: script returned exit code 1" followed by "Finished: FAILURE", with no further diagnostic text. (`jenkins.console_failures[2], line 3345; jenkins.console_tail`) - **jenkins** — The console log is truncated in the retained evidence, so the roughly 2900 lines between the scanner errors (line 399) and the final exit-code-1 line (line 3345) are not available. (`jenkins.console_truncated = true`) - **opensearch** — No application log records could be retrieved for the incident window 08:54:00Z-09:26:57Z; the log query failed with "opensearch request failed: [Errno 111] Connection refused". (`log_evidence.error`) ## Evidence The earliest failure region of the build console: ``` 09:02:50.252 INFO Sensor JaCoCo XML Report Importer [jacoco] (done) | time=60ms 09:02:50.253 INFO Sensor IaC CloudFormation Sensor [iac] 09:02:50.491 INFO 0 source files to be analyzed 09:02:50.642 INFO 0/0 source files have been analyzed 09:02:50.643 INFO Sensor IaC CloudFormation Sensor [iac] (done) | time=390ms 09:02:50.644 INFO Sensor IaC Kubernetes Sensor [iac] 09:02:50.870 INFO 2 source files to be analyzed 09:02:51.412 INFO 2/2 source files have been analyzed 09:02:51.450 INFO Sensor IaC Kubernetes Sensor [iac] (done) | time=806ms 09:02:51.450 INFO Sensor TypeScript analysis [javascript] 09:03:07.710 ERROR Error when running: 'node -v'. Is Node.js available during analysis? org.sonarsource.nodejs.NodeCommandException: Error when running: 'node -v'. Is Node.js available during analysis? at org.sonarsource.nodejs.NodeCommand.start(NodeCommand.java:79) at org.sonarsource.nodejs.NodeCommandBuilderImpl.getVersion(NodeCommandBuilderImpl.java:203) at org.sonarsource.nodejs.NodeCommandBuilderImpl.checkNodeCompatibility(NodeCommandBuilderImpl.java:169) at org.sonarsource.nodejs.NodeCommandBuilderImpl.build(NodeCommandBuilderImpl.java:143) at org.sonar.plugins.javascrip ... (truncated; the full log is in the linked build) ``` ## Inferences - Nothing in Soteria's test suite failed. No test report was published and no failing test is recorded, and the stage that failed is the quality-gate stage, not a test stage. - The Go analyzer failure is environmental, not a source-code problem. All 26 Go files failed identically with exit status 2 from an external helper process, including cmd/soteria/main.go. A real syntax error would affect specific files, not every file in the repository, and the same tree compiles ... - Node.js is not installed in the scanner container image. "Cannot run program \"node\": error=2, No such file or directory" is a missing executable on PATH, not a network, DNS, or registry problem, so re-running the same commit on the same image will reproduce it exactly. - Because zero Go files were analyzed and no coverage report was imported, the analysis submitted for this commit contained effectively none of the repository's measurements. The subsequent quality-gate check therefore evaluated a project with empty or zero coverage/issue data, which is the most pl... - A fix belongs in the CI scanner image and pipeline rather than in Soteria's source: the scanner container needs a Node.js runtime on PATH and a Go analyzer that can execute on the agent's architecture (or the scan must run on a matching architecture). Separately, the scan step should fail immedia... ## Links - Failed build: https://ci.bstein.dev/job/soteria/330/ - Full evidence bundle and audit trail live in Ariadne at `/api/admin/audit/events`, event types `hermes_autotriage_incident` and `hermes_autotriage_diagnosis`. Filed automatically by Ariadne from a Hermes Agent diagnosis (run [run_5b7d8e8a6b3e4860902ba7e637fd98a8](https://triage.bstein.dev/chat?resume=run_5b7d8e8a6b3e4860902ba7e637fd98a8)). Hermes has no write access to this repository; no files or infrastructure were changed. <!-- hermes-triage job=soteria classification=sonarqube_scanner_toolchain_missing incident=soteria/330 -->
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
atlas-reconciler bstein hermes-automation
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: titan/soteria#18
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?