Design Patterns
45 production AI design patterns, organized by the problem they solve.
Named, reusable solutions to the problems that show up once an AI system meets real traffic. Every pattern states the failure it prevents, when to reach for it, when not to, and what it costs you — across 9 categories from RAG and tool calling to multi-agent architectures and voice.
45 results
RAG & Retrieval
Hybrid Search with Reranking
Combine lexical and vector retrieval to cast a wide net, then let a cross-encoder decide what actually goes in the prompt.
RAG & Retrieval
Contextual Chunk Augmentation
Prepend document-level context to each chunk before embedding it, so an isolated passage still knows what it is about.
RAG & Retrieval
Query Rewriting and Decomposition
Transform the user's question into the queries your index can actually answer — resolved, expanded, and split into parts.
RAG & Retrieval
Corrective Retrieval Loop
Grade what you retrieved before you generate from it, and retrieve again — differently — when it is not good enough.
RAG & Retrieval
Grounded Citation and Refusal
Make every claim traceable to a retrieved span, and make "the documents don't say" a first-class, testable output.
Tool Calling & Function Execution
Tool Schema as Interface Design
Treat every tool definition as prompt engineering: the schema is what the model reads to decide whether, when, and how to call it.
Tool Calling & Function Execution
Parallel Tool Fan-Out
Let the model request several independent tools at once and execute them concurrently instead of walking them one turn at a time.
Tool Calling & Function Execution
Error as Observation
Feed tool failures back into the loop as structured, actionable text so the model can correct course instead of the run dying.
Tool Calling & Function Execution
Human-in-the-Loop Approval Gate
Classify tools by reversibility and require explicit human approval before the model takes an action it cannot take back.
Tool Calling & Function Execution
Dynamic Tool Selection
Retrieve a small relevant subset of tools per turn instead of pasting a catalog of two hundred into every request.
Multi-Agent Architectures
Orchestrator–Worker
A lead agent decomposes a task, dispatches independent subtasks to fresh workers, and synthesizes their returns into one answer.
Multi-Agent Architectures
Router Dispatch
Classify the request first, then hand it to one specialist configured for exactly that job.
Multi-Agent Architectures
Planner–Executor–Critic
Separate deciding what to do, doing it, and judging whether it worked — so the judgment is not made by the thing being judged.
Multi-Agent Architectures
Blackboard Shared State
Give agents a shared external store to read and write instead of passing state through each other's message histories.
Multi-Agent Architectures
Handoff and Escalation
Let an agent transfer the whole conversation to a better-suited agent or a human, carrying forward what it already learned.
Voice & Realtime AI
Cascaded vs. Speech-to-Speech
The first architectural decision in any voice product: three composable models in a chain, or one model that hears and speaks directly.
Voice & Realtime AI
Turn-Taking and Barge-In
Decide when the user has finished speaking and what happens when they talk over you — the two mechanics that make a voice agent feel human or broken.
Voice & Realtime AI
Latency Budget to First Audio
Treat time-to-first-audible-syllable as the product metric, and engineer every stage against an explicit allocation.
Voice & Realtime AI
Async Tools While Holding the Floor
Keep talking while a slow tool runs in the background, and weave its result in when it lands.
Voice & Realtime AI
Readback Confirmation
Read critical values back in a disambiguated form and get explicit confirmation before acting on anything transcribed.
Context & Memory
Context Budgeting
Give every component of the prompt an explicit token allocation and enforce it, instead of letting whatever arrives first fill the window.
Context & Memory
Progressive Summarization
Compress the oldest part of a conversation into a structured summary when it crosses a threshold, keeping recent turns verbatim.
Context & Memory
Working vs. Durable Memory
Separate the scratch state a task needs right now from the facts that should outlive the session, and store each differently.
Context & Memory
Retrieval-Augmented Memory
When durable facts outgrow the prompt, index them and retrieve only what this turn needs — with recency and importance in the ranking.
Context & Memory
Context Isolation via Subagents
Send noisy, exploratory work to a subagent with a disposable context and keep only its conclusion in the main thread.
Structured Output & Extraction
Schema-First Generation
Define the output type first and constrain decoding to it, so parsing becomes a guarantee rather than a hope.
Structured Output & Extraction
Validate and Repair Loop
When output fails validation, hand the model the specific errors and let it fix them instead of retrying blind.
Structured Output & Extraction
Extraction with Provenance
Return the source span and a confidence alongside every extracted field, so reviewers can verify without re-reading the document.
Structured Output & Extraction
Classification with an Escape Hatch
Give every classifier an explicit way to say "none of these" and "I am not sure", then route those cases instead of forcing a label.
Structured Output & Extraction
Decomposed Extraction
Split a large schema or a long document into focused passes instead of asking for eighty fields in one call.
Reliability & Guardrails
Input and Output Guardrail Chain
Run ordered, independently-failing checks before the model sees input and before the user sees output.
Reliability & Guardrails
Circuit Breaker and Fallback Chain
Detect a failing dependency, stop calling it, and degrade through a chain of progressively simpler alternatives.
Reliability & Guardrails
Idempotent, Replayable Runs
Checkpoint each step and key every side effect, so a crashed or retried run resumes instead of restarting or double-charging.
Reliability & Guardrails
Budget and Loop Ceilings
Bound every agent run by steps, tokens, wall clock, and spend — and stop with a useful partial result rather than an exception.
Reliability & Guardrails
Least-Privilege Tool Sandboxing
Scope every tool to the acting user's own authority, and assume any content the model reads may be trying to control it.
Cost, Latency & Routing
Model Cascade
Try the cheap model first, verify the answer, and escalate to a stronger one only when the cheap answer is not good enough.
Cost, Latency & Routing
Prompt and Semantic Caching
Cache at two levels: the stable prompt prefix the provider can reuse, and whole responses keyed by meaning rather than exact text.
Cost, Latency & Routing
Task-Based Model Routing
Match each task type to the cheapest model that clears its quality bar, decided by evaluation rather than by habit.
Cost, Latency & Routing
Streaming and Perceived Latency
Optimize what the user experiences as waiting — first token, visible progress, useful partial output — not total completion time.
Cost, Latency & Routing
Batching and Async Queues
Move work nobody is waiting on off the request path and into batches that are cheaper and easier to control.
Evaluation & Feedback Loops
Golden Set Regression Gate
Block any prompt, model, or pipeline change that regresses a curated set of known-correct cases.
Evaluation & Feedback Loops
LLM-as-Judge with a Calibrated Rubric
Score subjective quality with a model, but only against an explicit rubric that has been calibrated against human labels.
Evaluation & Feedback Loops
Online Feedback Capture
Instrument the product so real usage produces labelled data, then turn that data into evaluation cases and fixes.
Evaluation & Feedback Loops
Shadow and Canary Evaluation
Run the new version against real traffic without showing it, then expose it to a small slice before rolling it out.
Evaluation & Feedback Loops
Trace-Driven Failure Triage
Record every stage of every request as a structured trace, then attribute failures to the stage that actually caused them.
Browse by category
RAG & Retrieval
Getting the right context in front of the model — and knowing when there isn't any.
5 patterns
Tool Calling & Function Execution
Giving a model hands: schema design, fan-out, failure handling, and approval gates.
5 patterns
Multi-Agent Architectures
Shapes for splitting work across agents — and the coordination cost each one buys.
5 patterns
Voice & Realtime AI
Conversation under a latency budget: turn-taking, barge-in, and holding the floor.
5 patterns
Context & Memory
Deciding what the model sees this turn, and what survives to the next one.
5 patterns
Structured Output & Extraction
Turning generated text into data your system can actually depend on.
5 patterns
Reliability & Guardrails
Containing the blast radius when a model, a tool, or a provider misbehaves.
5 patterns
Cost, Latency & Routing
Making a system affordable and fast without giving up the quality you shipped for.
5 patterns
Evaluation & Feedback Loops
Knowing whether a change helped, before and after it reaches real traffic.
5 patterns