← All patterns
Cost, Latency & Routing
Making a system affordable and fast without giving up the quality you shipped for.
5 patterns
Model Cascade
Try the cheap model first, verify the answer, and escalate to a stronger one only when the cheap answer is not good enough.
8 minaka escalation ladder
Prompt and Semantic Caching
Cache at two levels: the stable prompt prefix the provider can reuse, and whole responses keyed by meaning rather than exact text.
9 minaka prefix caching
Task-Based Model Routing
Match each task type to the cheapest model that clears its quality bar, decided by evaluation rather than by habit.
8 minaka model selection
Streaming and Perceived Latency
Optimize what the user experiences as waiting — first token, visible progress, useful partial output — not total completion time.
8 minaka time to first token
Batching and Async Queues
Move work nobody is waiting on off the request path and into batches that are cheaper and easier to control.
8 minaka batch inference