titan/soteria
3
0
Fork 0
Code Issues 23 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

[hermes] soteria #657: transient_infra_failure #26

New Issue
Open
opened 2026-09-20 08:48:58 +00:00 by hermes-automation · 0 comments
hermes-automation commented 2026-09-20 08:48:58 +00:00
Member
Copy Link

Hermes auto-triage classified incident soteria/657 as transient_infra_failure (confidence 0.95); first failed gate: scm_checkout.

Why a human is needed

Build 657 never got past fetching the source: the Git server at scm.bstein.dev answered the fetch of titan/soteria.git with HTTP 500, Jenkins exhausted its checkout retries, and the build aborted about 2.5 seconds in, before any stage, code, or test ran. Nothing in this repository caused the failure and nothing in it needs changing. The path to green is a healthy response from the Git server followed by a re-run of the same commit; if 500s continue on retry, the Gitea host itself needs operator attention.

Ariadne did not authorize automated remediation: evidence_signature_missing.

Facts

  • jenkins — Build soteria #657 finished FAILURE after 2.468 seconds (started 2026-09-20T08:45:45Z, ended 08:45:47Z). (https://ci.bstein.dev/job/soteria/657/ (result, timestamps))
  • jenkins — The git fetch performed while checking out the repository to read the Jenkinsfile exited with status 128: fatal: unable to access 'https://scm.bstein.dev/titan/soteria.git/': The requested URL returned error: 500. (console_failures[0] at line 13; also present in console_tail)
  • jenkins — Jenkins retried the checkout and aborted with 'ERROR: Maximum checkout retry attempts reached, aborting'; the failure occurred during the Jenkinsfile checkout, before any pipeline stage started. (console_tail)
  • jenkins — No pipeline stage was reported (first_failed_stage is null) and no test results were published (failed_tests is empty). (jenkins.first_failed_stage, jenkins.failed_tests)
  • jenkins — The fetch used the 'Gitea PAT for pipelines' credential via GIT_ASKPASS before the 500 was returned. (console_tail line 'using GIT_ASKPASS to set credentials Gitea PAT for pipelines')
  • opensearch — Correlated log evidence is unavailable for this window: the OpenSearch query failed with '[Errno 111] Connection refused' and returned zero records. (log_evidence.error)

Evidence

The earliest failure region of the build console:

	at PluginClassLoader for git//hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1239)
	at PluginClassLoader for git//hudson.plugins.git.GitSCM._checkout(GitSCM.java:1310)
	at PluginClassLoader for git//hudson.plugins.git.GitSCM.checkout(GitSCM.java:1277)
	at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:136)
	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:168)
	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:70)
	at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:317)
	at hudson.model.ResourceController.execute(ResourceController.java:101)
	at hudson.model.Executor.run(Executor.java:460)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://scm.bstein.dev/titan/soteria.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: fatal: unable to access 'https://scm.bstein.dev/titan/soteria.git/': The requested URL returned error: 5
... (truncated; the full log is in the linked build)

Inferences

  • The failure is on the Git server side: HTTP 500 is a server error from scm.bstein.dev, and an authentication or missing-repository problem would have surfaced as 401/403/404 instead. Nothing from the soteria repository was checked out or executed, so the service's code and tests played no part in...
  • The 2.5-second duration together with the absence of any stage or published test result confirms the build died at its very first step, fetching the repository to read the Jenkinsfile.
  • A fix involves confirming the Gitea service behind scm.bstein.dev is responding normally and re-running the build for the same commit; if 500 responses persist across re-runs, the Git server itself (service, database, or storage) needs investigation, not this repository.
  • The OpenSearch connection refusal is a gap in evidence collection for this incident, not part of the build failure; no log records were available to corroborate the Gitea 500.

Links

  • Failed build: https://ci.bstein.dev/job/soteria/657/
  • Full evidence bundle and audit trail live in Ariadne at /api/admin/audit/events, event types hermes_autotriage_incident and hermes_autotriage_diagnosis.

Filed automatically by Ariadne from a Hermes Agent diagnosis (run run_69dd9ba0946b461e89f4eaaa721aeafc). Hermes has no write access to this repository; no files or infrastructure were changed.

Hermes auto-triage classified incident `soteria/657` as **transient_infra_failure** (confidence 0.95); first failed gate: `scm_checkout`. ## Why a human is needed Build 657 never got past fetching the source: the Git server at scm.bstein.dev answered the fetch of titan/soteria.git with HTTP 500, Jenkins exhausted its checkout retries, and the build aborted about 2.5 seconds in, before any stage, code, or test ran. Nothing in this repository caused the failure and nothing in it needs changing. The path to green is a healthy response from the Git server followed by a re-run of the same commit; if 500s continue on retry, the Gitea host itself needs operator attention. Ariadne did not authorize automated remediation: `evidence_signature_missing`. ## Facts - **jenkins** — Build soteria #657 finished FAILURE after 2.468 seconds (started 2026-09-20T08:45:45Z, ended 08:45:47Z). (`https://ci.bstein.dev/job/soteria/657/ (result, timestamps)`) - **jenkins** — The git fetch performed while checking out the repository to read the Jenkinsfile exited with status 128: fatal: unable to access 'https://scm.bstein.dev/titan/soteria.git/': The requested URL returned error: 500. (`console_failures[0] at line 13; also present in console_tail`) - **jenkins** — Jenkins retried the checkout and aborted with 'ERROR: Maximum checkout retry attempts reached, aborting'; the failure occurred during the Jenkinsfile checkout, before any pipeline stage started. (`console_tail`) - **jenkins** — No pipeline stage was reported (first_failed_stage is null) and no test results were published (failed_tests is empty). (`jenkins.first_failed_stage, jenkins.failed_tests`) - **jenkins** — The fetch used the 'Gitea PAT for pipelines' credential via GIT_ASKPASS before the 500 was returned. (`console_tail line 'using GIT_ASKPASS to set credentials Gitea PAT for pipelines'`) - **opensearch** — Correlated log evidence is unavailable for this window: the OpenSearch query failed with '[Errno 111] Connection refused' and returned zero records. (`log_evidence.error`) ## Evidence The earliest failure region of the build console: ``` at PluginClassLoader for git//hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1239) at PluginClassLoader for git//hudson.plugins.git.GitSCM._checkout(GitSCM.java:1310) at PluginClassLoader for git//hudson.plugins.git.GitSCM.checkout(GitSCM.java:1277) at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:136) at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:168) at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:70) at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:317) at hudson.model.ResourceController.execute(ResourceController.java:101) at hudson.model.Executor.run(Executor.java:460) Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://scm.bstein.dev/titan/soteria.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: fatal: unable to access 'https://scm.bstein.dev/titan/soteria.git/': The requested URL returned error: 5 ... (truncated; the full log is in the linked build) ``` ## Inferences - The failure is on the Git server side: HTTP 500 is a server error from scm.bstein.dev, and an authentication or missing-repository problem would have surfaced as 401/403/404 instead. Nothing from the soteria repository was checked out or executed, so the service's code and tests played no part in... - The 2.5-second duration together with the absence of any stage or published test result confirms the build died at its very first step, fetching the repository to read the Jenkinsfile. - A fix involves confirming the Gitea service behind scm.bstein.dev is responding normally and re-running the build for the same commit; if 500 responses persist across re-runs, the Git server itself (service, database, or storage) needs investigation, not this repository. - The OpenSearch connection refusal is a gap in evidence collection for this incident, not part of the build failure; no log records were available to corroborate the Gitea 500. ## Links - Failed build: https://ci.bstein.dev/job/soteria/657/ - Full evidence bundle and audit trail live in Ariadne at `/api/admin/audit/events`, event types `hermes_autotriage_incident` and `hermes_autotriage_diagnosis`. Filed automatically by Ariadne from a Hermes Agent diagnosis (run [run_69dd9ba0946b461e89f4eaaa721aeafc](https://triage.bstein.dev/chat?resume=run_69dd9ba0946b461e89f4eaaa721aeafc)). Hermes has no write access to this repository; no files or infrastructure were changed. <!-- hermes-triage job=soteria classification=transient_infra_failure incident=soteria/657 -->
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
atlas-reconciler bstein hermes-automation
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: titan/soteria#26
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?