Router Dispatch
Also known as: intent routing, triage agent, specialist dispatch
Classify the request first, then hand it to one specialist configured for exactly that job.
The problem
A single generalist agent handling every request carries every instruction, every tool, and every edge case in one prompt. Billing rules dilute technical troubleshooting guidance, the tool list grows past the point of reliable selection, and improving one domain's behaviour regresses another's. The prompt becomes a contested resource that no one can safely edit.
The pattern
Put a cheap fast classifier in front. It reads the request, picks one of a small set of named destinations, and forwards. Each destination is a separate agent with its own system prompt, its own tools, and its own model tier. The router itself does no domain work and holds no domain knowledge beyond what distinguishes the categories.
Use when
- Requests fall into a handful of distinct domains with genuinely different handling
- Different categories warrant different model tiers, tools, or safety policies
- Your single prompt has grown past the point where changes are safe to make
- You want per-domain evaluation and independent deployment of behaviours
Avoid when
- Requests routinely span categories — routing forces a false choice and loses half the task
- There are only two or three destinations with mostly shared behaviour
- Misroute cost is high and your categories are not cleanly separable
Tradeoffs
GainEach specialist stays small, testable, and independently improvable
CostMisroutes are the dominant failure and they are invisible from inside a specialist
GainCheap models handle cheap categories; expensive ones only where needed
CostA classification hop on every request, adding latency before any work starts
GainPer-domain tools mean smaller tool lists and better selection accuracy
CostShared behaviour must be factored out or it drifts apart across specialists
Routing is the cheapest multi-agent pattern and often the only one a product needs. It is also the one most likely to be over-engineered into a mesh of agents talking to agents, when what was wanted was a switch statement with good classification.
Make routing a classification, not a conversation
The router should emit a single label from a closed set, plus a confidence. Give it a small fast model, a tight prompt, and one-line descriptions of each destination that emphasize the boundary between them. Always include a fallback destination so an unclassifiable request has somewhere to land — routing to a general handler is recoverable, refusing to route is not.
class Route(BaseModel):
destination: Literal["billing", "technical", "account", "sales", "general"]
confidence: float = Field(ge=0, le=1)
reason: str
ROUTER_SYSTEM = """Classify the user's request into exactly one destination.
billing — charges, invoices, refunds, payment methods, plan pricing
technical — errors, integrations, API behaviour, anything not working
account — login, permissions, seats, profile and org settings
sales — evaluating the product, upgrades, quotes, contracts
general — anything else, or when two destinations fit equally well
Classify the request, not the tone. An angry message about an invoice is billing."""
def route(message: str, history: list[Message]) -> Route:
return structured_call(
ROUTER_SYSTEM, user=render(history[-3:] + [message]),
schema=Route, model="small", temperature=0,
)Dispatch with a low-confidence path
A confidence score is only useful if something acts on it. Below a threshold, route to the general handler rather than committing to a specialist that will confidently answer the wrong question. Log every low-confidence route — that log is the highest-value source of new category definitions you will find.
SPECIALISTS = {
"billing": Agent(prompt=BILLING, tools=BILLING_TOOLS, model="small"),
"technical": Agent(prompt=TECHNICAL, tools=TECH_TOOLS, model="large"),
"account": Agent(prompt=ACCOUNT, tools=ACCOUNT_TOOLS, model="small"),
"sales": Agent(prompt=SALES, tools=CRM_TOOLS, model="small"),
"general": Agent(prompt=GENERAL, tools=READ_ONLY_TOOLS, model="large"),
}
MIN_CONFIDENCE = 0.6
async def handle(message: str, conv: Conversation) -> Reply:
r = route(message, conv.history)
chosen = r.destination if r.confidence >= MIN_CONFIDENCE else "general"
if chosen != r.destination:
metrics.increment("router.low_confidence", tags={"intended": r.destination})
log.info("low confidence route", extra={"reason": r.reason, "message": message})
conv.route = chosen # sticky: the next turn stays put unless intent shifts
return await SPECIALISTS[chosen].respond(message, conv)Make routing sticky within a conversation. Re-classifying every turn causes an agent to change personality mid-thread when a follow-up is short and ambiguous. Re-route only on an explicit topic shift, or when the current specialist says it cannot help.
Let specialists reroute
The router will misclassify. Give each specialist a `transfer` tool so it can hand off when it discovers the request is not its domain, passing a short summary of what it already learned so the user is not asked the same questions twice. That converts a misroute from a dead end into one extra hop — and the transfer log becomes your router's training data.
- Closed label set, small model, temperature zero, always with a fallback destination
- Act on confidence: below threshold, prefer the generalist over a confident wrong specialist
- Make the route sticky per conversation; re-route on topic shift, not every turn
- Give specialists a transfer tool that carries context forward
- Mine low-confidence routes and transfers for the categories you are missing
Do not let the router answer. The moment it starts handling easy cases itself, it accumulates domain knowledge, its prompt grows, and you have rebuilt the generalist you were trying to split up — with an extra hop in front of it.