fix(hermes): cli-auto capacity failover uses automatic Switchyard reclassification #32

Merged

bstein merged 2 commits from fix/cli-auto-failover-effort into main 2026-08-21 23:13:09 +00:00

Conversation 1 Commits 2 Files Changed 5 +164 -16

hermes-automation commented 2026-08-21 05:08:01 +00:00

Member

Copy Link

Summary

cli-auto capacity/auth/quota failover previously called select_route with a
hardcoded manual lane (cli-{alternate}-{effort}), so the retry boundary was
classified switchyard-manual instead of going through automatic Jetson
classification — misleading routing evidence, even though effort happened to
be preserved as a side effect of the hardcoded lane string.

services/hermes/scripts/cli_lane_failover.py now retries through Switchyard
automatic selection: select_route(context, assignee, exclude_provider=route.provider, exclude_reason=...)
with assignee still cli-auto. If the classifier reclassifies to a lower
effort than the original route (including when its own excluded-provider
health guard fires), the lane re-pins the chosen provider at the original
effort floor via one manual-lane call, so a capacity failure never silently
downgrades a high/xhigh task. Effort is allowed to escalate if the classifier
picks something higher. Explicit manual lanes (assignee other than cli-auto)
are unchanged and still fail closed without switching providers. Both-hosted
failure still blocks transient and never falls to a local or metered path.

What changed

• services/hermes/scripts/cli_lane_failover.py — automatic reclassification
  with provider exclusion, plus effort-floor preservation/escalation.
• testing/tests/test_hermes_cli_quota_failover.py — new coverage for
  automatic classification on retry, the exclude_provider/exclude_reason
  kwargs, and effort-floor escalation when the classifier downgrades; manual
  fail-closed tests kept and unaffected.
• testing/tests/test_hermes_cli_fallback.py,
  testing/tests/test_hermes_cli_review_lane.py,
  testing/tests/test_hermes_cli_router_outage.py — updated stubs so the
  retry select_route call is distinguished by the exclusion kwarg instead of
  a hardcoded manual lane string, matching the new call shape. Two of these
  tests were also isolated onto tmp-path provider-health files instead of the
  real /opt/data/provider-health/*.json, closing a latent test-isolation gap
  that the old hardcoded-lane stubs happened to mask.

Evidence

• New tests fail against the pre-change implementation (confirmed red before
  implementing) and pass after.
• Focused run: pytest testing/tests/test_hermes_cli_quota_failover.py testing/tests/test_hermes_cli_fallback.py testing/tests/test_hermes_cli_router_outage.py testing/tests/test_hermes_cli_review_lane.py testing/tests/test_hermes_cli_lane_routing.py testing/tests/test_hermes_cli_routing.py testing/tests/test_hermes_cli_lane_goal.py — all pass.
• Full suite: pytest testing/tests/ — 2538 passed, 4 skipped, 4 failed. The 4
  failures (test_hermes_agent_layout.py x2,
  test_hermes_gitea_pr_integration.py::test_http_error_path_redacts_token,
  test_hermes_kanban_supervisor.py::test_auto_supervise_flag_defaults_false_in_configmap)
  reproduce identically on unmodified origin/main, confirmed by stashing
  this change and re-running them — unrelated pre-existing failures.
• No manifest or kustomize wiring changed, so no kustomize build services/hermes was needed.

Known related gap (not fixed here, out of scope)

services/hermes/scripts/execution_pool_worker.py (the newer two-worker
concurrency execution path) has the same hardcoded-manual-lane pattern at its
own capacity-failure retry call. This PR only touches
cli_lane_failover.py, the file named in the task. Flagging it here for a
follow-up.

Test plan

• pytest testing/tests/test_hermes_cli_quota_failover.py (new + updated
  coverage)
• pytest testing/tests/test_hermes_cli_fallback.py testing/tests/test_hermes_cli_router_outage.py testing/tests/test_hermes_cli_review_lane.py testing/tests/test_hermes_cli_lane_routing.py testing/tests/test_hermes_cli_routing.py testing/tests/test_hermes_cli_lane_goal.py
• pytest testing/tests/ full suite (only pre-existing unrelated
  failures)
• Not merged/reconciled/deployed per task instructions — draft only.

## Summary `cli-auto` capacity/auth/quota failover previously called `select_route` with a hardcoded manual lane (`cli-{alternate}-{effort}`), so the retry boundary was classified `switchyard-manual` instead of going through automatic Jetson classification — misleading routing evidence, even though effort happened to be preserved as a side effect of the hardcoded lane string. `services/hermes/scripts/cli_lane_failover.py` now retries through Switchyard automatic selection: `select_route(context, assignee, exclude_provider=route.provider, exclude_reason=...)` with `assignee` still `cli-auto`. If the classifier reclassifies to a lower effort than the original route (including when its own excluded-provider health guard fires), the lane re-pins the chosen provider at the original effort floor via one manual-lane call, so a capacity failure never silently downgrades a high/xhigh task. Effort is allowed to escalate if the classifier picks something higher. Explicit manual lanes (assignee other than `cli-auto`) are unchanged and still fail closed without switching providers. Both-hosted failure still blocks transient and never falls to a local or metered path. ## What changed - `services/hermes/scripts/cli_lane_failover.py` — automatic reclassification with provider exclusion, plus effort-floor preservation/escalation. - `testing/tests/test_hermes_cli_quota_failover.py` — new coverage for automatic classification on retry, the `exclude_provider`/`exclude_reason` kwargs, and effort-floor escalation when the classifier downgrades; manual fail-closed tests kept and unaffected. - `testing/tests/test_hermes_cli_fallback.py`, `testing/tests/test_hermes_cli_review_lane.py`, `testing/tests/test_hermes_cli_router_outage.py` — updated stubs so the retry `select_route` call is distinguished by the exclusion kwarg instead of a hardcoded manual lane string, matching the new call shape. Two of these tests were also isolated onto tmp-path provider-health files instead of the real `/opt/data/provider-health/*.json`, closing a latent test-isolation gap that the old hardcoded-lane stubs happened to mask. ## Evidence - New tests fail against the pre-change implementation (confirmed red before implementing) and pass after. - Focused run: `pytest testing/tests/test_hermes_cli_quota_failover.py testing/tests/test_hermes_cli_fallback.py testing/tests/test_hermes_cli_router_outage.py testing/tests/test_hermes_cli_review_lane.py testing/tests/test_hermes_cli_lane_routing.py testing/tests/test_hermes_cli_routing.py testing/tests/test_hermes_cli_lane_goal.py` — all pass. - Full suite: `pytest testing/tests/` — 2538 passed, 4 skipped, 4 failed. The 4 failures (`test_hermes_agent_layout.py` x2, `test_hermes_gitea_pr_integration.py::test_http_error_path_redacts_token`, `test_hermes_kanban_supervisor.py::test_auto_supervise_flag_defaults_false_in_configmap`) reproduce identically on unmodified `origin/main`, confirmed by stashing this change and re-running them — unrelated pre-existing failures. - No manifest or kustomize wiring changed, so no `kustomize build services/hermes` was needed. ## Known related gap (not fixed here, out of scope) `services/hermes/scripts/execution_pool_worker.py` (the newer two-worker concurrency execution path) has the same hardcoded-manual-lane pattern at its own capacity-failure retry call. This PR only touches `cli_lane_failover.py`, the file named in the task. Flagging it here for a follow-up. ## Test plan - [x] `pytest testing/tests/test_hermes_cli_quota_failover.py` (new + updated coverage) - [x] `pytest testing/tests/test_hermes_cli_fallback.py testing/tests/test_hermes_cli_router_outage.py testing/tests/test_hermes_cli_review_lane.py testing/tests/test_hermes_cli_lane_routing.py testing/tests/test_hermes_cli_routing.py testing/tests/test_hermes_cli_lane_goal.py` - [x] `pytest testing/tests/` full suite (only pre-existing unrelated failures) - [ ] Not merged/reconciled/deployed per task instructions — draft only.

hermes-automation added 1 commit 2026-08-21 05:08:02 +00:00

fix(hermes): cli-auto capacity failover uses automatic Switchyard reclassification ... d579c08cbe

Capacity/auth/quota failover for cli-auto previously called select_route
with a hardcoded manual lane (cli-{alternate}-{effort}), so the retry
boundary was classified as switchyard-manual instead of going through
Jetson automatic classification, making the routing evidence misleading.

Now the retry calls select_route(context, "cli-auto", exclude_provider=...)
so the boundary stays automatically classified with an explicit
failed-provider exclusion. If the classifier reclassifies to a lower
effort than the original route, the lane re-pins the chosen provider at
the original effort floor so a capacity failure never silently downgrades
a high/xhigh task. Manual lanes (assignee != cli-auto) remain unchanged
and still fail closed without switching providers.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

bstein added 1 commit 2026-08-21 23:12:14 +00:00

Merge branch 'main' into fix/cli-auto-failover-effort 77905ff1db

bstein changed title from WIP: fix(hermes): cli-auto capacity failover uses automatic Switchyard reclassification to fix(hermes): cli-auto capacity failover uses automatic Switchyard reclassification 2026-08-21 23:12:22 +00:00

bstein approved these changes 2026-08-21 23:13:02 +00:00

bstein merged commit 0841681a06 into main 2026-08-21 23:13:09 +00:00

bstein referenced this issue from a commit 2026-08-21 23:13:09 +00:00

Merge pull request 'fix(hermes): cli-auto capacity failover uses automatic Switchyard reclassification' (#32) from fix/cli-auto-failover-effort into main

Sign in to join this conversation.

Reviewers

No Reviewers

bstein

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

atlas-reconciler bstein hermes-automation

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: titan/atlas-iac#32

No description provided.