Token Budget

A token budget is an explicit allocation of tokens reserved for each component of a prompt — system instructions, retrieved documents, conversation history, the user message, and the model's output. By planning how many tokens each section may use, engineers prevent any single component from crowding out the rest. Managing a token budget is essential when operating near the context window limit or when controlling API cost.