titan/soteria
3
0
Fork 0
Code Issues 87 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

[hermes] soteria #765: transient_infra_failure #32

New Issue
Open
opened 2026-09-22 23:31:55 +00:00 by hermes-automation · 0 comments
hermes-automation commented 2026-09-22 23:31:55 +00:00
Member
Copy Link

Hermes auto-triage classified incident soteria/765 as transient_infra_failure (confidence 0.99); first failed gate: SCM checkout to load Jenkinsfile, before pipeline execution.

Why a human is needed

Soteria build 765 failed because fetching the repository to load its Jenkinsfile received HTTP 500 from the SCM endpoint. Jenkins aborted after exhausting checkout retries; no Soteria test assertion or quality gate caused this failure. Re-run the same commit after SCM availability recovers. No repository code or test change is indicated; recurring HTTP 500 responses would require investigation of the SCM service and its proxy.

Ariadne did not authorize automated remediation: evidence_signature_missing.

Facts

  • jenkins — Jenkins was checking out https://scm.bstein.dev/titan/soteria.git to read the Jenkinsfile when the failure occurred. (bundle.jenkins.console_tail: opening checkout message and CpsScmFlowDefinition.create stack frames)
  • jenkins — The earliest failure region shows git fetch exiting with status 128 because the SCM HTTP request returned 500: 'RPC failed; HTTP 500 curl 22'. Git then reported 'fatal: expected flush after ref listing'. (bundle.jenkins.console_failures[0], marker at line 20 and its nested GitException)
  • jenkins — Jenkins exhausted its checkout retry attempts and ended the build with FAILURE. (bundle.jenkins.console_failures[0]: 'Maximum checkout retry attempts reached, aborting'; bundle.jenkins.result)
  • jenkins — No failed pipeline stage or structured failed tests were reported. (bundle.jenkins.first_failed_stage = null; bundle.jenkins.failed_tests = [])
  • opensearch — The supplementary OpenSearch lookup failed with connection refused and returned no records. (bundle.log_evidence.error; bundle.log_evidence.records)

Evidence

The earliest failure region of the build console:

	at PluginClassLoader for git//hudson.plugins.git.GitSCM._checkout(GitSCM.java:1310)
	at PluginClassLoader for git//hudson.plugins.git.GitSCM.checkout(GitSCM.java:1277)
	at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:136)
	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:168)
	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:70)
	at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:317)
	at hudson.model.ResourceController.execute(ResourceController.java:101)
	at hudson.model.Executor.run(Executor.java:460)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://scm.bstein.dev/titan/soteria.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500
fatal: expected flush after ref listing

	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPII
... (truncated; the full log is in the linked build)

Inferences

  • The build failed during Jenkinsfile retrieval, before Soteria's pipeline gates or tests could execute. The Git protocol error follows the HTTP 500; it does not establish a repository source defect.
  • The HTTP 500 points to the SCM service or an intermediary serving its requests. Retrying the same commit after that endpoint recovers should be sufficient; the bundle does not establish the server-side cause or confirm recovery.
  • The OpenSearch connection refusal limits corroborating evidence but does not establish a connection to the SCM failure.

Links

  • Failed build: https://ci.bstein.dev/job/soteria/765/
  • Full evidence bundle and audit trail live in Ariadne at /api/admin/audit/events, event types hermes_autotriage_incident and hermes_autotriage_diagnosis.

Filed automatically by Ariadne from a Hermes Agent diagnosis (run run_d00bdf9ad4444585950b4031ad7e75f9). Hermes has no write access to this repository; no files or infrastructure were changed.

Hermes auto-triage classified incident `soteria/765` as **transient_infra_failure** (confidence 0.99); first failed gate: `SCM checkout to load Jenkinsfile, before pipeline execution`. ## Why a human is needed Soteria build 765 failed because fetching the repository to load its Jenkinsfile received HTTP 500 from the SCM endpoint. Jenkins aborted after exhausting checkout retries; no Soteria test assertion or quality gate caused this failure. Re-run the same commit after SCM availability recovers. No repository code or test change is indicated; recurring HTTP 500 responses would require investigation of the SCM service and its proxy. Ariadne did not authorize automated remediation: `evidence_signature_missing`. ## Facts - **jenkins** — Jenkins was checking out https://scm.bstein.dev/titan/soteria.git to read the Jenkinsfile when the failure occurred. (`bundle.jenkins.console_tail: opening checkout message and CpsScmFlowDefinition.create stack frames`) - **jenkins** — The earliest failure region shows git fetch exiting with status 128 because the SCM HTTP request returned 500: 'RPC failed; HTTP 500 curl 22'. Git then reported 'fatal: expected flush after ref listing'. (`bundle.jenkins.console_failures[0], marker at line 20 and its nested GitException`) - **jenkins** — Jenkins exhausted its checkout retry attempts and ended the build with FAILURE. (`bundle.jenkins.console_failures[0]: 'Maximum checkout retry attempts reached, aborting'; bundle.jenkins.result`) - **jenkins** — No failed pipeline stage or structured failed tests were reported. (`bundle.jenkins.first_failed_stage = null; bundle.jenkins.failed_tests = []`) - **opensearch** — The supplementary OpenSearch lookup failed with connection refused and returned no records. (`bundle.log_evidence.error; bundle.log_evidence.records`) ## Evidence The earliest failure region of the build console: ``` at PluginClassLoader for git//hudson.plugins.git.GitSCM._checkout(GitSCM.java:1310) at PluginClassLoader for git//hudson.plugins.git.GitSCM.checkout(GitSCM.java:1277) at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:136) at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:168) at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:70) at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:317) at hudson.model.ResourceController.execute(ResourceController.java:101) at hudson.model.Executor.run(Executor.java:460) Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://scm.bstein.dev/titan/soteria.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500 fatal: expected flush after ref listing at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPII ... (truncated; the full log is in the linked build) ``` ## Inferences - The build failed during Jenkinsfile retrieval, before Soteria's pipeline gates or tests could execute. The Git protocol error follows the HTTP 500; it does not establish a repository source defect. - The HTTP 500 points to the SCM service or an intermediary serving its requests. Retrying the same commit after that endpoint recovers should be sufficient; the bundle does not establish the server-side cause or confirm recovery. - The OpenSearch connection refusal limits corroborating evidence but does not establish a connection to the SCM failure. ## Links - Failed build: https://ci.bstein.dev/job/soteria/765/ - Full evidence bundle and audit trail live in Ariadne at `/api/admin/audit/events`, event types `hermes_autotriage_incident` and `hermes_autotriage_diagnosis`. Filed automatically by Ariadne from a Hermes Agent diagnosis (run [run_d00bdf9ad4444585950b4031ad7e75f9](https://triage.bstein.dev/chat?resume=run_d00bdf9ad4444585950b4031ad7e75f9)). Hermes has no write access to this repository; no files or infrastructure were changed. <!-- hermes-triage job=soteria classification=transient_infra_failure incident=soteria/765 -->
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
atlas-reconciler bstein hermes-automation
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: titan/soteria#32
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?