Prompt Compression

Also known as: context compression

Prompt compression reduces the token count of a prompt while preserving the information needed for accurate responses. Techniques range from simple heuristics like removing boilerplate and redundant examples to neural methods that distill long contexts into shorter summaries or soft token representations. Compression is especially important in long-document QA and multi-turn conversations where the raw context can easily overflow the model's context window or drive up inference costs.