bstein/atlasbot
1
0
Fork 0
Code Issues 160 Pull Requests 5 Actions Packages Projects Releases Wiki Activity

[hermes] atlasbot #413: transient_infra_failure #11

New Issue
Open
opened 2026-09-21 23:20:06 +00:00 by hermes-automation · 0 comments
hermes-automation commented 2026-09-21 23:20:06 +00:00
Copy Link

Hermes auto-triage classified incident atlasbot/413 as transient_infra_failure (confidence 0.9); first failed gate: scm_checkout.

Why a human is needed

Build 413 never obtained the source: while Jenkins was fetching bstein/atlasbot from scm.bstein.dev to read the Jenkinsfile, the Git server answered the ref listing with HTTP 500 and truncated the response, Jenkins exhausted its checkout retries, and the run ended in about 25 seconds before any stage or test executed. Nothing in the atlasbot repository caused this and no code change is needed; the commit is simply untested. Re-running the same commit once scm.bstein.dev responds normally resolves it, and if 500s persist across retries the Gitea service itself (pod, database, or storage) is what needs attention.

Ariadne did not authorize automated remediation: evidence_signature_missing.

Facts

  • jenkins — Build atlasbot #413 ran from 2026-09-21T23:12:37Z to 2026-09-21T23:13:02Z (24.9s) and finished FAILURE. (https://ci.bstein.dev/job/atlasbot/413/)
  • jenkins — The failure occurred during the initial git fetch of https://scm.bstein.dev/bstein/atlasbot.git performed to read the Jenkinsfile: git exited with status 128, stderr 'error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500' and 'fatal: expected flush after ref listing'. (jenkins.console_failures[0] (marker ERROR:, line 13))
  • jenkins — Jenkins retried the checkout and aborted with 'ERROR: Maximum checkout retry attempts reached, aborting'; the build reported no pipeline stage (jenkins.first_failed_stage is null) and published no test results (jenkins.failed_tests is empty). (jenkins.console_tail; jenkins.first_failed_stage; jenkins.failed_tests)
  • jenkins — Credential setup completed before the failing fetch: the console shows 'using GIT_ASKPASS to set credentials Gitea PAT for pipelines' immediately before the git fetch command. (jenkins.console_tail)
  • opensearch — No correlated log records are available for the incident window: the OpenSearch query failed with 'opensearch request failed: [Errno 111] Connection refused' and returned zero records. (log_evidence.error, query_window 2026-09-21T23:07:37Z to 2026-09-21T23:18:02Z)

Evidence

The earliest failure region of the build console:

	at PluginClassLoader for git//hudson.plugins.git.GitSCM._checkout(GitSCM.java:1310)
	at PluginClassLoader for git//hudson.plugins.git.GitSCM.checkout(GitSCM.java:1277)
	at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:136)
	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:168)
	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:70)
	at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:317)
	at hudson.model.ResourceController.execute(ResourceController.java:101)
	at hudson.model.Executor.run(Executor.java:460)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://scm.bstein.dev/bstein/atlasbot.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500
fatal: expected flush after ref listing

	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAP
... (truncated; the full log is in the linked build)

Inferences

  • The HTTP 500 was returned by the Git server at scm.bstein.dev while it was advertising refs; 'fatal: expected flush after ref listing' means the server cut the smart-HTTP response short mid-stream, which points at a server-side fault on the SCM host rather than at anything in the atlasbot reposit...
  • Because the build failed before the repository was ever checked out, no atlasbot code, tests, or even the Jenkinsfile were executed; the commit that triggered this build is untested rather than broken.
  • Re-running the build against the same commit should succeed once scm.bstein.dev is serving normally again. If repeated runs keep hitting 500s, the underlying problem is in the Gitea service itself (its pod, database, or storage), and its server-side logs for this window would show the real error;...

Links

  • Failed build: https://ci.bstein.dev/job/atlasbot/413/
  • Full evidence bundle and audit trail live in Ariadne at /api/admin/audit/events, event types hermes_autotriage_incident and hermes_autotriage_diagnosis.

Filed automatically by Ariadne from a Hermes Agent diagnosis (run run_eff24eaab9fa4aad94e7f66f27b20929). Hermes has no write access to this repository; no files or infrastructure were changed.

Hermes auto-triage classified incident `atlasbot/413` as **transient_infra_failure** (confidence 0.9); first failed gate: `scm_checkout`. ## Why a human is needed Build 413 never obtained the source: while Jenkins was fetching bstein/atlasbot from scm.bstein.dev to read the Jenkinsfile, the Git server answered the ref listing with HTTP 500 and truncated the response, Jenkins exhausted its checkout retries, and the run ended in about 25 seconds before any stage or test executed. Nothing in the atlasbot repository caused this and no code change is needed; the commit is simply untested. Re-running the same commit once scm.bstein.dev responds normally resolves it, and if 500s persist across retries the Gitea service itself (pod, database, or storage) is what needs attention. Ariadne did not authorize automated remediation: `evidence_signature_missing`. ## Facts - **jenkins** — Build atlasbot #413 ran from 2026-09-21T23:12:37Z to 2026-09-21T23:13:02Z (24.9s) and finished FAILURE. (`https://ci.bstein.dev/job/atlasbot/413/`) - **jenkins** — The failure occurred during the initial git fetch of https://scm.bstein.dev/bstein/atlasbot.git performed to read the Jenkinsfile: git exited with status 128, stderr 'error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500' and 'fatal: expected flush after ref listing'. (`jenkins.console_failures[0] (marker ERROR:, line 13)`) - **jenkins** — Jenkins retried the checkout and aborted with 'ERROR: Maximum checkout retry attempts reached, aborting'; the build reported no pipeline stage (jenkins.first_failed_stage is null) and published no test results (jenkins.failed_tests is empty). (`jenkins.console_tail; jenkins.first_failed_stage; jenkins.failed_tests`) - **jenkins** — Credential setup completed before the failing fetch: the console shows 'using GIT_ASKPASS to set credentials Gitea PAT for pipelines' immediately before the git fetch command. (`jenkins.console_tail`) - **opensearch** — No correlated log records are available for the incident window: the OpenSearch query failed with 'opensearch request failed: [Errno 111] Connection refused' and returned zero records. (`log_evidence.error, query_window 2026-09-21T23:07:37Z to 2026-09-21T23:18:02Z`) ## Evidence The earliest failure region of the build console: ``` at PluginClassLoader for git//hudson.plugins.git.GitSCM._checkout(GitSCM.java:1310) at PluginClassLoader for git//hudson.plugins.git.GitSCM.checkout(GitSCM.java:1277) at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:136) at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:168) at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:70) at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:317) at hudson.model.ResourceController.execute(ResourceController.java:101) at hudson.model.Executor.run(Executor.java:460) Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://scm.bstein.dev/bstein/atlasbot.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500 fatal: expected flush after ref listing at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAP ... (truncated; the full log is in the linked build) ``` ## Inferences - The HTTP 500 was returned by the Git server at scm.bstein.dev while it was advertising refs; 'fatal: expected flush after ref listing' means the server cut the smart-HTTP response short mid-stream, which points at a server-side fault on the SCM host rather than at anything in the atlasbot reposit... - Because the build failed before the repository was ever checked out, no atlasbot code, tests, or even the Jenkinsfile were executed; the commit that triggered this build is untested rather than broken. - Re-running the build against the same commit should succeed once scm.bstein.dev is serving normally again. If repeated runs keep hitting 500s, the underlying problem is in the Gitea service itself (its pod, database, or storage), and its server-side logs for this window would show the real error;... ## Links - Failed build: https://ci.bstein.dev/job/atlasbot/413/ - Full evidence bundle and audit trail live in Ariadne at `/api/admin/audit/events`, event types `hermes_autotriage_incident` and `hermes_autotriage_diagnosis`. Filed automatically by Ariadne from a Hermes Agent diagnosis (run [run_eff24eaab9fa4aad94e7f66f27b20929](https://triage.bstein.dev/chat?resume=run_eff24eaab9fa4aad94e7f66f27b20929)). Hermes has no write access to this repository; no files or infrastructure were changed. <!-- hermes-triage job=atlasbot classification=transient_infra_failure incident=atlasbot/413 -->
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
bstein
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: bstein/atlasbot#11
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?