Decomposed Extraction
Also known as: multi-pass extraction, map-reduce over documents, field grouping
Split a large schema or a long document into focused passes instead of asking for eighty fields in one call.
The problem
Extraction accuracy degrades as schemas grow. A model asked for eighty fields in one pass attends unevenly: early fields are accurate, later ones get progressively sloppier, and fields requiring careful reading are answered carelessly because they compete with seventy-nine others for attention. Long documents compound this — the relevant text for field sixty may sit forty pages from the text for field two, and a single pass reads neither carefully.
The pattern
Decompose along whichever axis is causing the problem. Split the schema into coherent field groups, each extracted by a focused call with only the instructions that group needs. Split the document into sections and extract from each, then reduce. Group fields that share a source region so each pass reads one part of the document well, and run the passes in parallel.
Use when
- The schema exceeds roughly twenty fields, or contains groups needing different instructions
- Documents are long enough that relevant text for different fields is far apart
- Accuracy visibly degrades for fields later in a large schema
- Different field groups need different models — cheap for simple, strong for judgment
Avoid when
- The schema is small and single-pass accuracy is already good
- Fields are deeply interdependent and must be reasoned about together
- Per-call overhead dominates — many tiny passes over a short document is pure waste
Tradeoffs
GainPer-field accuracy improves substantially, especially for later and subtler fields
CostN calls instead of one; document tokens are re-sent per pass unless cached
GainPasses run in parallel, so wall-clock time can be lower than the single big call
CostMerging results and resolving cross-group inconsistency is new logic to own
GainEach group can use the model tier its difficulty warrants
CostMore moving parts to evaluate, monitor, and keep consistent
The observation behind this pattern is simple: a model does better on ten fields than on eighty, and better on the section that contains the answer than on a hundred pages that mostly do not. Decomposition buys back both.
Group fields by what they need
Good groups share a source region, a required expertise, and a difficulty level. Party details come from the first page; financial terms come from the schedule; obligations come from the body and need careful reading. Each group gets its own focused instructions instead of one prompt trying to cover every field's quirks at once.
@dataclass
class Group:
name: str
schema: type[BaseModel]
instructions: str
model: str
sections: list[str] | None = None # None means the whole document
GROUPS = [
Group("parties", Parties, "Extract the legal entities and signatories.",
model="small", sections=["preamble", "signature"]),
Group("financial", FinancialTerms,
"Extract amounts as integer cents. Never infer a figure that is not printed.",
model="large", sections=["schedule", "payment"]),
Group("obligations", Obligations,
"Extract each party's duties. One entry per distinct obligation.",
model="large"),
Group("dates", KeyDates, "Extract dates in ISO format. Distinguish effective from execution date.",
model="small"),
]Run the passes in parallel, over the right text
Because groups are independent, they run concurrently — so N passes cost roughly one pass in wall-clock time. Narrowing each pass to its relevant sections cuts token cost too, but keep a whole-document fallback: a section classifier that guesses wrong would otherwise silently blank a group.
async def extract(document: Document) -> Contract:
async def run(g: Group) -> tuple[str, BaseModel]:
text = document.sections(g.sections) if g.sections else document.full_text
if len(text) < MIN_USEFUL_CHARS:
text = document.full_text # section routing missed; do not blank the group
return g.name, await structured_call(
system=g.instructions, user=text, schema=g.schema, model=g.model
)
results = dict(await asyncio.gather(*(run(g) for g in GROUPS)))
return merge(results)Cache the document prefix across passes. Every pass sends the same source text, so provider-side prompt caching turns N full-price passes into one full-price pass plus N cheap ones — which removes most of the cost objection to this pattern.
Map-reduce for long documents
When a single field's evidence is scattered across a long document — every obligation in a hundred-page contract — split by section, extract from each, and reduce. The reduce step is not concatenation: it must deduplicate near-identical items, resolve conflicting values, and keep provenance so a merged item still points at its source.
async def extract_all_obligations(doc: Document) -> list[Obligation]:
per_section = await asyncio.gather(*(
structured_call(system=OBLIGATIONS, user=s.text, schema=ObligationList)
for s in doc.sections_of_type("body")
))
merged: list[Obligation] = []
for section_result in per_section:
for item in section_result.obligations:
twin = find_near_duplicate(merged, item, threshold=0.88)
if twin is None:
merged.append(item)
else:
twin.sources.extend(item.sources) # same duty, stated twice
return mergedReconcile across groups
Independent passes can disagree — the dates group and the financial group may each report a different effective date. Do not silently prefer one. Run cross-group validation after the merge and treat conflicts the way you treat any low-confidence field: flag for review rather than picking arbitrarily.
- Group fields by source region, expertise, and difficulty — not alphabetically
- Run groups in parallel; wall-clock stays close to a single pass
- Keep a whole-document fallback when section routing yields too little text
- Cache the document prefix so repeated passes are cheap
- Validate across groups after merging and flag conflicts rather than resolving silently
Measure per-group accuracy separately. Aggregate accuracy hides the case where one group is near-perfect and another is failing badly — and the aggregate number will look acceptable right up until someone relies on the broken group.