Prompt Caching

Prompt caching stores the processed representation of a stable prompt prefix — such as a long system prompt or shared context — so repeated requests reuse it instead of recomputing. This cuts both latency and cost for workloads that send the same preamble many times. Unlike semantic caching, which reuses whole responses, prompt caching accelerates the prefill stage of otherwise distinct requests.