Progressive Summarization

The failure to design against is an agent that forgets a constraint from turn three and cheerfully violates it at turn forty. Generic prose summaries are prone to exactly that, because they optimize for readability rather than for retaining obligations.

Summarize into a schema, not into prose

A structured summary forces the model to enumerate the categories that matter and makes omissions visible. It also composes: folding an old summary into a new one is a merge of typed lists rather than a re-write of paragraphs, which limits how much drift accumulates.

class Summary(BaseModel):
    objective: str                  # what the user is ultimately trying to do
    decisions: list[str]            # settled choices — do not revisit these
    constraints: list[str]          # rules stated by the user; still binding
    established_facts: list[str]    # verified values: ids, amounts, dates
    open_threads: list[str]         # raised and not yet resolved
    rejected: list[str]             # approaches already tried and ruled out


SUMMARIZE = """Compress this conversation segment into the schema.

Preserve exactly: decisions, user-stated constraints, verified facts with their
values, unresolved questions, and approaches already rejected (with why).
Discard: pleasantries, restatements, and reasoning that led to a recorded decision.
Never generalize a specific value into a vague one. If a prior summary is given,
merge it in — carry forward everything still relevant, drop only what is settled
and superseded."""

Compact on a threshold, keep a verbatim tail

Trigger compaction when history crosses a fraction of the window, not on every turn. Always keep the last several turns verbatim — the agent needs precise recent context to continue coherently — and never compact across a tool call and its result, since a call without its result is a dangling reference most providers reject.

COMPACT_AT = 0.65        # fraction of the history budget
KEEP_VERBATIM = 8        # most recent turns, always exact


def maybe_compact(state: ConvState, budget: int) -> ConvState:
    if count_tokens(state.messages) < budget * COMPACT_AT:
        return state

    split = safe_split_point(state.messages, keep_last=KEEP_VERBATIM)
    old, recent = state.messages[:split], state.messages[split:]
    if not old:
        return state

    state.summary = structured_call(
        SUMMARIZE,
        user=render(old, prior_summary=state.summary),
        schema=Summary,
    )
    state.messages = recent
    metrics.increment("context.compaction")
    return state


def safe_split_point(messages: list[Message], keep_last: int) -> int:
    """Never split a tool call from its result — a dangling call is a hard error."""
    i = max(0, len(messages) - keep_last)
    while i < len(messages) and messages[i].role == "tool":
        i += 1
    return i

Rendering the summary back in

Place the summary at the top of the reconstructed context, clearly labelled as a compressed record rather than as something the user said. Marking constraints and decisions as still binding matters — without that framing, models treat summarized content as background colour and feel free to revisit settled questions.

def render_context(state: ConvState) -> list[Message]:
    if not state.summary:
        return state.messages

    s = state.summary
    header = (
        "[Earlier conversation, compressed. Decisions and constraints below remain "
        "binding — do not revisit them unless the user reopens them.]\n\n"
        f"Objective: {s.objective}\n"
        f"Decisions: {bullets(s.decisions)}\n"
        f"Constraints: {bullets(s.constraints)}\n"
        f"Established: {bullets(s.established_facts)}\n"
        f"Already rejected: {bullets(s.rejected)}\n"
        f"Open: {bullets(s.open_threads)}"
    )
    return [Message(role="system", content=header), *state.messages]

• Summarize into a schema with explicit slots for decisions, constraints, and facts
• Keep the most recent turns verbatim; never split a tool call from its result
• Fold the previous summary into the new one rather than re-summarizing raw history
• Label the summary as binding, not as background
• Archive the full transcript outside the context — compaction is for the model, not for storage