hermes: enforce worker routing quality floors
All checks were successful
Tests / Declarative: Post Actions passed: 244
All checks were successful
Tests / Declarative: Post Actions passed: 244
This commit is contained in:
parent
85b78405dd
commit
d81778013f
@ -336,16 +336,60 @@ data:
|
|||||||
tool_calling = false
|
tool_calling = false
|
||||||
reasoning = true
|
reasoning = true
|
||||||
prompt = """
|
prompt = """
|
||||||
You are the sole routing authority for one durable engineering-worker
|
You are the deterministic routing authority for one durable engineering
|
||||||
launch. Choose exactly one configured worker target from the current
|
worker launch. Choose exactly one configured target. Apply these steps in
|
||||||
objective, acceptance criteria, risk, and any prior-provider failure in the
|
order.
|
||||||
request. Prefer Codex for implementation, debugging, tests, and repository
|
|
||||||
changes. Prefer Claude for architecture, ambiguity, synthesis, adversarial
|
1. Set a mandatory minimum effort floor from the whole objective:
|
||||||
analysis, and independent review. Use low only for mechanical bounded work,
|
- xhigh: critical security work; a risky production migration; destructive
|
||||||
medium for ordinary work, high for difficult or consequential work, and
|
or data-loss risk; or an independent final/release review of consequential
|
||||||
xhigh for critical security, migration, destructive-risk, or independent
|
changes.
|
||||||
final review. Never exceed xhigh. If the request says a provider has failed
|
- high: difficult debugging, consequential implementation, adversarial
|
||||||
or exhausted capacity, do not select that provider.
|
review, architecture with major tradeoffs, or uncertain production work.
|
||||||
|
- medium: ordinary implementation, tests, analysis, or bounded architecture
|
||||||
|
work.
|
||||||
|
- low: only mechanical, reversible, tightly bounded work such as a typo,
|
||||||
|
formatting, or a simple lookup.
|
||||||
|
Never choose below the floor and never exceed xhigh.
|
||||||
|
|
||||||
|
2. Determine provider availability before provider preference:
|
||||||
|
- Anthropic and Claude name the same provider. If either is failed,
|
||||||
|
unavailable, exhausted, rate-limited, or out of capacity, Claude is
|
||||||
|
unavailable.
|
||||||
|
- OpenAI and Codex name the same provider. If either is failed, unavailable,
|
||||||
|
exhausted, rate-limited, or out of capacity, Codex is unavailable.
|
||||||
|
- Never select an unavailable provider. Use the available provider at the
|
||||||
|
same effort floor.
|
||||||
|
|
||||||
|
3. Choose the provider for the dominant action when both are available:
|
||||||
|
- Codex: implementation, debugging, tests, commands, and repository changes.
|
||||||
|
- Claude: architecture, ambiguity, synthesis, risk analysis, adversarial
|
||||||
|
analysis, and independent review.
|
||||||
|
A final independent review is Claude; implementing review findings is Codex.
|
||||||
|
|
||||||
|
4. Map provider and effort exactly:
|
||||||
|
Codex low=worker_codex_luna_low;
|
||||||
|
Codex medium=worker_codex_terra_medium;
|
||||||
|
Codex high=worker_codex_sol_high;
|
||||||
|
Codex xhigh=worker_codex_sol_xhigh.
|
||||||
|
Claude low=worker_claude_haiku_low;
|
||||||
|
Claude medium=worker_claude_sonnet_medium;
|
||||||
|
Claude high=worker_claude_sonnet_high;
|
||||||
|
Claude xhigh=worker_claude_opus_xhigh.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
Critical security migration final review -> worker_claude_opus_xhigh.
|
||||||
|
Implement critical security review fixes -> worker_codex_sol_xhigh.
|
||||||
|
Difficult intermittent production failure -> worker_codex_sol_high.
|
||||||
|
Ordinary component design -> worker_claude_sonnet_medium.
|
||||||
|
One spelling correction -> worker_codex_luna_low.
|
||||||
|
Anthropic exhausted + critical independent final review ->
|
||||||
|
worker_codex_sol_xhigh.
|
||||||
|
OpenAI exhausted + difficult repository implementation ->
|
||||||
|
worker_claude_sonnet_high.
|
||||||
|
|
||||||
|
Before responding, verify the provider is available and effort is not below
|
||||||
|
the floor. Return only the required decision object.
|
||||||
"""
|
"""
|
||||||
response_schema = '''
|
response_schema = '''
|
||||||
{"type":"object","properties":{"decision":{"type":"object","properties":{"target":{"type":"string","enum":["worker_codex_sol_high","worker_claude_sonnet_high","worker_codex_sol_xhigh","worker_claude_opus_xhigh","worker_codex_terra_medium","worker_claude_sonnet_medium","worker_codex_luna_low","worker_claude_haiku_low"]}},"required":["target"],"additionalProperties":false}},"required":["decision"],"additionalProperties":false}
|
{"type":"object","properties":{"decision":{"type":"object","properties":{"target":{"type":"string","enum":["worker_codex_sol_high","worker_claude_sonnet_high","worker_codex_sol_xhigh","worker_claude_opus_xhigh","worker_codex_terra_medium","worker_claude_sonnet_medium","worker_codex_luna_low","worker_claude_haiku_low"]}},"required":["target"],"additionalProperties":false}},"required":["decision"],"additionalProperties":false}
|
||||||
|
|||||||
@ -19,7 +19,7 @@ spec:
|
|||||||
labels:
|
labels:
|
||||||
app: hermes-switchyard
|
app: hermes-switchyard
|
||||||
annotations:
|
annotations:
|
||||||
ai.bstein.dev/config-rev: "20260811-switchyard-authority-v8"
|
ai.bstein.dev/config-rev: "20260811-switchyard-authority-v9"
|
||||||
prometheus.io/scrape: "true"
|
prometheus.io/scrape: "true"
|
||||||
prometheus.io/port: "9005"
|
prometheus.io/port: "9005"
|
||||||
prometheus.io/path: /metrics
|
prometheus.io/path: /metrics
|
||||||
|
|||||||
@ -769,6 +769,10 @@ def test_local_flux_runtime_and_gpu_handoff_are_flux_managed():
|
|||||||
assert 'id = "qwen2.5:14b-instruct-q4_0"' in switchyard
|
assert 'id = "qwen2.5:14b-instruct-q4_0"' in switchyard
|
||||||
assert "qwen2.5:3b-instruct-q4_0" not in switchyard
|
assert "qwen2.5:3b-instruct-q4_0" not in switchyard
|
||||||
assert "route/local/qwen2.5-14b/medium" in switchyard
|
assert "route/local/qwen2.5-14b/medium" in switchyard
|
||||||
|
assert "Codex xhigh=worker_codex_sol_xhigh" in switchyard
|
||||||
|
assert "Claude xhigh=worker_claude_opus_xhigh" in switchyard
|
||||||
|
assert "Anthropic and Claude name the same provider" in switchyard
|
||||||
|
assert "OpenAI and Codex name the same provider" in switchyard
|
||||||
model_gate = _documents(HERMES / "model-gate-configmap.yaml")[0]["data"][
|
model_gate = _documents(HERMES / "model-gate-configmap.yaml")[0]["data"][
|
||||||
"model_gate.py"
|
"model_gate.py"
|
||||||
]
|
]
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user