[hermes] ananke #353: scm_repository_not_found #196

New Issue

hermes-automation · 2026-09-25T18:14:27Z

hermes-automation commented

2026-09-25 18:14:27 +00:00

Hermes auto-triage classified incident ananke/353 as scm_repository_not_found (confidence 0.92); first failed gate: Declarative: Checkout SCM.

Why a human is needed

Build 353 never obtained source: with its normal credential presented, Gitea answered the clone of https://scm.bstein.dev/titan/ananke.git with '404 page not found / repository not found', and the build aborted after exhausting checkout retries, so nothing was compiled or tested. That same URL answers successfully now and the old bstein/ananke location returns a permanent redirect, which points to the repository having been transferred or renamed right around the build window — the build most likely raced the move. A maintainer should confirm the transfer to titan/ananke was intentional and complete, verify the Jenkins job definition, webhooks, and any clone or submodule references use the final titan/ananke path, and then re-run the same commit; it should now check out cleanly. If a re-run still returns 404, the pipeline token's access to the repository's new owner is the next thing to check, since Gitea shows 404 for repositories a credential cannot see. Only a maintainer can confirm the move was intended and trigger that re-run, which is why this needs a human.

Ariadne did not authorize automated remediation: human_required.

Facts

jenkins — Build ananke #353 (2026-09-25T17:57:00Z to 18:07:21Z) finished FAILURE, and the first failed pipeline stage was 'Declarative: Checkout SCM'; no later stage ran. (https://ci.bstein.dev/job/ananke/353/ — jenkins.result, jenkins.first_failed_stage)
jenkins — The checkout failed with git exit status 128: 'remote: 404 page not found' and "fatal: repository 'https://scm.bstein.dev/titan/ananke.git/' not found", while authenticating via GIT_ASKPASS with the 'Gitea PAT for pipelines' credential; Jenkins then aborted with 'Maximum checkout retry attempts r... (jenkins.console_failures, marker 'ERROR:' at console line 194, and jenkins.console_tail)
jenkins — No structured test results were published (jenkins.failed_tests is empty), consistent with the build failing before any compile or test step executed. (jenkins.failed_tests)
gitea — At triage time, after the build, GET https://scm.bstein.dev/api/v1/repos/titan/ananke returns HTTP 200 and GET https://scm.bstein.dev/api/v1/repos/bstein/ananke returns HTTP 301: the repository now exists at the exact path that returned 404 during the build, and the old bstein/ananke path issues ... (unauthenticated Gitea API probes, 2026-09-25 after 18:08 UTC)
opensearch — Centralized log correlation for the incident window was unavailable (OpenSearch request failed: connection refused), so the Jenkins console is the only log evidence for this build. (log_evidence.error)

Evidence

The earliest failure region of the build console:

tty: true
    volumeMounts:
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
  - command:
    - "cat"
    image: "registry.bstein.dev/bstein/quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64"
    name: "quality-tools"
    tty: true
    volumeMounts:
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
  - env:
    - name: "JENKINS_SECRET"
      value: "********"
    - name: "JENKINS_TUNNEL"
      value: "jenkins.jenkins.svc.cluster.local:50000"
    - name: "JENKINS_AGENT_NAME"
      value: "ananke-353-9bjgb-3k8bd-5pgp1"

Inferences

The 404 during the build, combined with the same URL answering successfully shortly afterwards and the old bstein/ananke path now permanently redirecting, indicates the repository was being transferred or renamed around 17:57–18:07 UTC and the build raced the move; the failure is about repository...
Because checkout never completed, the commit was never exercised; the same commit is expected to check out and build normally now that the repository answers at titan/ananke.
A residual alternative is that the pipeline token could not yet see the repository under its new owner at build time, since Gitea reports 404 rather than 403 for repositories a credential cannot view; a re-run is the fastest discriminator, and if it still 404s the token's access to the titan owne...
The trailing 'Required context class hudson.FilePath is missing' error in the post actions is downstream noise from failing before any workspace existed (archiveArtifacts runs in an always post block with no node context); guarding that archiving step so it is skipped when no workspace was alloca...

Suggested remediation (not available)

No automated remediation exists for this failure. Hermes proposes one; it was not performed, and Ariadne cannot perform it until a person builds and deploys it.

Proposed action: rerun_build_after_scm_repo_restored
What it would do: Probe the exact clone URL from the failing build's console with the job's SCM credential and, once it answers successfully again, re-run the failed build at the same commit, because a checkout 404 means the commit was never exercised and the rebuild is the entire fix.
Evidence that should be required first: The build's console must show git checkout failing with a 404 / 'repository not found' from the SCM host before any build or test step ran, and a current authenticated probe of that same clone URL must return success, proving the repository is reachable again rather than deleted or the job URL being permanently wrong.

Links

Failed build: https://ci.bstein.dev/job/ananke/353/
Full evidence bundle and audit trail live in Ariadne at /api/admin/audit/events, event types hermes_autotriage_incident and hermes_autotriage_diagnosis.

Filed automatically by Ariadne from a Hermes Agent diagnosis (run run_31311e0eab744afba78001d07144b704). Hermes has no write access to this repository; no files or infrastructure were changed.

Hermes auto-triage classified incident `ananke/353` as **scm_repository_not_found** (confidence 0.92); first failed gate: `Declarative: Checkout SCM`. ## Why a human is needed Build 353 never obtained source: with its normal credential presented, Gitea answered the clone of https://scm.bstein.dev/titan/ananke.git with '404 page not found / repository not found', and the build aborted after exhausting checkout retries, so nothing was compiled or tested. That same URL answers successfully now and the old bstein/ananke location returns a permanent redirect, which points to the repository having been transferred or renamed right around the build window — the build most likely raced the move. A maintainer should confirm the transfer to titan/ananke was intentional and complete, verify the Jenkins job definition, webhooks, and any clone or submodule references use the final titan/ananke path, and then re-run the same commit; it should now check out cleanly. If a re-run still returns 404, the pipeline token's access to the repository's new owner is the next thing to check, since Gitea shows 404 for repositories a credential cannot see. Only a maintainer can confirm the move was intended and trigger that re-run, which is why this needs a human. Ariadne did not authorize automated remediation: `human_required`. ## Facts - **jenkins** — Build ananke #353 (2026-09-25T17:57:00Z to 18:07:21Z) finished FAILURE, and the first failed pipeline stage was 'Declarative: Checkout SCM'; no later stage ran. (`https://ci.bstein.dev/job/ananke/353/ — jenkins.result, jenkins.first_failed_stage`) - **jenkins** — The checkout failed with git exit status 128: 'remote: 404 page not found' and "fatal: repository 'https://scm.bstein.dev/titan/ananke.git/' not found", while authenticating via GIT_ASKPASS with the 'Gitea PAT for pipelines' credential; Jenkins then aborted with 'Maximum checkout retry attempts r... (`jenkins.console_failures, marker 'ERROR:' at console line 194, and jenkins.console_tail`) - **jenkins** — No structured test results were published (jenkins.failed_tests is empty), consistent with the build failing before any compile or test step executed. (`jenkins.failed_tests`) - **gitea** — At triage time, after the build, GET https://scm.bstein.dev/api/v1/repos/titan/ananke returns HTTP 200 and GET https://scm.bstein.dev/api/v1/repos/bstein/ananke returns HTTP 301: the repository now exists at the exact path that returned 404 during the build, and the old bstein/ananke path issues ... (`unauthenticated Gitea API probes, 2026-09-25 after 18:08 UTC`) - **opensearch** — Centralized log correlation for the incident window was unavailable (OpenSearch request failed: connection refused), so the Jenkins console is the only log evidence for this build. (`log_evidence.error`) ## Evidence The earliest failure region of the build console: ``` tty: true volumeMounts: - mountPath: "/home/jenkins/agent" name: "workspace-volume" - command: - "cat" image: "registry.bstein.dev/bstein/quality-tools:sonar8.0.1-trivy0.70.0-db20260422-arm64" name: "quality-tools" tty: true volumeMounts: - mountPath: "/home/jenkins/agent" name: "workspace-volume" - env: - name: "JENKINS_SECRET" value: "********" - name: "JENKINS_TUNNEL" value: "jenkins.jenkins.svc.cluster.local:50000" - name: "JENKINS_AGENT_NAME" value: "ananke-353-9bjgb-3k8bd-5pgp1" ``` ## Inferences - The 404 during the build, combined with the same URL answering successfully shortly afterwards and the old bstein/ananke path now permanently redirecting, indicates the repository was being transferred or renamed around 17:57–18:07 UTC and the build raced the move; the failure is about repository... - Because checkout never completed, the commit was never exercised; the same commit is expected to check out and build normally now that the repository answers at titan/ananke. - A residual alternative is that the pipeline token could not yet see the repository under its new owner at build time, since Gitea reports 404 rather than 403 for repositories a credential cannot view; a re-run is the fastest discriminator, and if it still 404s the token's access to the titan owne... - The trailing 'Required context class hudson.FilePath is missing' error in the post actions is downstream noise from failing before any workspace existed (archiveArtifacts runs in an always post block with no node context); guarding that archiving step so it is skipped when no workspace was alloca... ## Suggested remediation (not available) No automated remediation exists for this failure. Hermes proposes one; it was not performed, and Ariadne cannot perform it until a person builds and deploys it. - Proposed action: `rerun_build_after_scm_repo_restored` - What it would do: Probe the exact clone URL from the failing build's console with the job's SCM credential and, once it answers successfully again, re-run the failed build at the same commit, because a checkout 404 means the commit was never exercised and the rebuild is the entire fix. - Evidence that should be required first: The build's console must show git checkout failing with a 404 / 'repository not found' from the SCM host before any build or test step ran, and a current authenticated probe of that same clone URL must return success, proving the repository is reachable again rather than deleted or the job URL being permanently wrong. ## Links - Failed build: https://ci.bstein.dev/job/ananke/353/ - Full evidence bundle and audit trail live in Ariadne at `/api/admin/audit/events`, event types `hermes_autotriage_incident` and `hermes_autotriage_diagnosis`. Filed automatically by Ariadne from a Hermes Agent diagnosis (run [run_31311e0eab744afba78001d07144b704](https://triage.bstein.dev/chat?resume=run_31311e0eab744afba78001d07144b704)). Hermes has no write access to this repository; no files or infrastructure were changed.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: titan/ananke#196