switchboardGitHub repo

Portfolio project: deterministic-first routing with a model fallback

switchboard

A git-native ticket router for a fleet of specialist AI agents

File a ticket; get connected to the right agent, or told honestly that none fit. Routing starts with free, instant tag-overlap scoring. A model is consulted only when no tag overlaps at all, and only if you pass --ai or --jev. Every human correction is recorded, and the --ai router sees the 10 most recent ones.

See what is measuredView the repoWhat is not built

  • 78 tests passing offline (counted locally)
  • Latest CI run passed (checked)
  • Tests mock the model (SWITCHBOARD_MOCK=1)
  • Jev backend tested offline only
  • Risk tiers are advisory
  • Livery comparison: author's own, unverified

01

The problem

Manually deciding "which agent should handle this" doesn't scale past a handful of agents, and asking an LLM every single time is slow and costs money for decisions that are usually obvious.

Switchboard's router is deterministic by default (instant, free, tag-overlap matching), falls back to a model only when no tag overlaps at all (and only if you pass --jev or --ai), refuses to force a bad match either way, and records every time a human corrects it.

  • A router that always answers

    A router that always answers trains you to stop checking the answer, so route_deterministic returns None when no tag overlaps instead of guessing.

  • Confidence has to gate the action

    Only medium or high AI confidence commits an assignment; low confidence is a recorded suggestion and the ticket stays open.

  • Fire-and-forget is a robustness gap

    --run uses Popen (PID captured at start) through to a DispatchAttempt record, proven against a real failure, not mocked.

02

How it works

A ticket is routed by tag overlap. Only on zero overlap, and only if you asked, does a model get consulted. A human can override any decision, and dispatch prints the command unless you pass --run.

  1. New ticketa markdown file with a title, tags and a body
  2. Tag-overlap routerscore above zero: assign and record the rationale; no API call
  3. Zero overlap: Jev, then Claudeonly with --jev or --ai; otherwise the ticket stays unrouted for a human
  4. Human correctionreroute records why; the last 10 corrections are shown to the --ai router only
  5. Dispatchprints the command by default; --run executes it and records a durable attempt
  6. Closeappended to ledger.md, an append-only trail

Stages, one at a time

Stage 1 Deterministic router

Scores every registered agent by tag overlap and accepts any score above zero. No API call, no network.

A weak one-tag overlap is routed without a model check. The model is consulted for "no match", not for ambiguity.

Stage 2 Model fallback, only when asked

If no agent shares a single tag, the ticket stays unrouted unless you pass --jev or --ai. Jev can abstain and Claude may say none fit. High or medium confidence assigns; low confidence is recorded as a suggestion and leaves the ticket unassigned.

Stage 3 Corrections

reroute lets a human override any decision and permanently records why. The 10 most recent corrections are shown to the --ai router on later calls.

The deterministic router ignores corrections. There is no fine-tuning or retraining.

Stage 4 Dispatch and close

Dispatch composes the exact shell command (or, for a claude_code agent, a live session prompt) and prints it by default. --run executes it for real with a durable attempt record. Closing a ticket appends it to ledger.md.

Risk tiers are advisory: a medium or high tier prints a WARNING on --run and never blocks it.

03

What is measured

There is no routing-accuracy benchmark. What is checked is the size of the project and two dispatch attempts shown in the README.

Project scale and test suite

Taken from the README header and the repository. The tests run offline with SWITCHBOARD_MOCK=1; real --ai, talk and debate calls need an API key.

ItemValue
Registered agents8 (one markdown file each)
Runtimes2 (a command string, or a live claude_code session)
Tests78
Python versions tested3.9 and 3.14

Caveat Counts of things that exist, not measures of routing quality. The 9-point self-comparison against Livery is the author's own and has not been independently verified.

Source: README header and "Running the tests"; agents/

Two real dispatch attempts (README)

Dispatch attempt records shown in the README as proof of the durable-attempt and live-runtime claims. Each is a single dispatch.

Ticket and agentStatusDetail
0002, tpm-agent-osfailedreturncode 2: the agent's repo was not cloned locally, and the failure was recorded instead of a silent hang
0006, research-assistantsucceededreturncode 0, a real session id, cost_usd 0.15030852000000003; read ARCHITECTURE.md with its Read tool and answered correctly

Caveat One dispatch each, run on 2026-08-31. The attempt JSON files are not committed; the records appear in the README, and ledger.md records the closing summary for ticket 0006.

Source: README "Real findings from building and testing this"; ledger.md

04

Architecture pattern

Pattern Router/Dispatcher. switchboard/router.py picks one agent for a ticket (route_deterministic, then optionally route_with_jev or route_with_ai), and switchboard/dispatch.py hands the ticket to that agent's runtime. Switchboard does the choosing and handing off; the registered agents do the work, and Switchboard does not run an agent loop or combine their outputs.

Routing starts with tag-overlap scoring and accepts any score above zero. A model is consulted only when no tag overlaps and the user passes --jev or --ai. Low model confidence is recorded as a suggestion and leaves the ticket unassigned. Dispatch itself is deterministic. It is a one-shot router, so there is no planning, no multi-step coordination between agents and no check on the quality of an agent's result.

  1. Ticketmarkdown plus tags
  2. Tag-overlap routerscore above zero: assign and record the rationale
  3. Jev router (opt-in)zero overlap and --jev; typed choice with probabilities, can abstain
  4. Claude router (opt-in)--ai; sees the last 10 corrections; low confidence is a suggestion only
  5. Human triage and rerouteappends a correction to memory/routing_corrections.jsonl
  6. Dispatchprints by default; --run records a DispatchAttempt with pid and status
  7. ledger.mdappend-only record of closed tickets

Design rationale in the repo

05

Competencies demonstrated

  • Deterministic-first system design

    Tag-overlap matching runs before any model call, so the common case is instant and free.

  • Graceful escalation

    A model is consulted only when no tag overlaps and --jev or --ai is passed, with an honest refusal path if nothing fits.

  • Feedback-loop design

    Every reroute is recorded; the 10 most recent are included in the --ai router's prompt.

  • Independent implementation

    Built from scratch against Livery's concept with no shared code.

  • Honest self-assessment

    The Livery comparison is the author's own and unverified, and it tries to state where this is behind.

06

What is not built or not verified yet

Taken from the repository's own README and its known limits. This is part of the pitch: the gaps are known, written down and not rounded away.

  • limit

    A model is rarely consulted

    It runs only on zero tag overlap and only with --ai or --jev. That means "no match", not a judgement of ambiguity, so a weak one-tag overlap is routed without a model check.

  • not verified

    Jev backend: offline only

    Tested with mock fixtures and stub clients. Live Jev behavior was verified on the standalone benchmark in jev-agent-router, not through Switchboard. No live Switchboard + Jev run has been done.

  • limit

    Risk tiers only warn

    A risk_tier of medium or high prints a WARNING on --run; it never blocks execution.

  • limit

    Correction memory is partial

    Only the last 10 corrections reach the --ai router's prompt. The deterministic router ignores them, and there is no fine-tuning or retraining.

  • limit

    Debate is one model

    The two-persona debate mode is one model playing both sides, not independent agents or runtimes.

  • limit

    One-shot router

    No planning, no multi-step coordination between agents and no check on the quality of an agent's result.

  • limit

    One real live runtime

    A single real adapter (claude -p). Codex, Cursor, LM Studio and Ollama would each need their own verified adapter.

  • not verified

    Comparison with Livery

    The table is the author's own and has not been independently verified against Livery's current behavior.

  • not built

    More schedule shapes

    Cron beyond "fixed time(s), daily" raises a clear error instead of guessing.

07

Roadmap

  • DoneAutomatic routing (deterministic plus AI fallback) and routing-correction memory
  • DoneDurable dispatch attempts and concurrency-safe ticket creation
  • DoneTalk mode, persona-adapted debate, scheduling (declared, rendered, installable behind --apply) and notifications
  • DoneA live claude_code runtime adapter, proven with a real dispatch
  • Not doneRuntime breadth beyond Claude Code: one real adapter beats four guessed ones
  • Not doneCron beyond "fixed time(s), daily"

08

  • muster

    The manual-assignment CLI baseline this repo adds automatic routing on top of, as an idea, not shared code.

  • taskloom

    Another take on the same interest from the visual desktop-app angle.

  • jev-agent-router

    Where live Jev behavior was verified on a standalone router benchmark.