Schema-First Generation
Also known as: constrained decoding, JSON mode, structured outputs
Define the output type first and constrain decoding to it, so parsing becomes a guarantee rather than a hope.
The problem
Asking a model for JSON in the prompt and parsing whatever comes back is a coin flip at scale. The model wraps the object in a code fence, prefixes it with "Here's the JSON you requested", invents a field, omits a required one, or emits a trailing comma. Each of these is individually rare and collectively common, and every one becomes a production exception in a pipeline that assumed the output was well-formed.
The pattern
Define the output as a schema in your own type system, derive the JSON Schema from it, and pass it to the provider's structured-output mode so the decoder can only emit tokens consistent with it. Parse into your type and let validation, not string inspection, be the contract. Where constrained decoding is unavailable, keep the schema and add a repair loop behind it.
Use when
- Model output feeds another system — a database write, an API call, a UI component
- You need specific fields present and correctly typed, every time
- Enumerations matter and invented values would be unsafe downstream
- You are currently regex-extracting JSON out of prose responses
Avoid when
- The output is prose meant for a human to read
- The schema would be so deeply nested that generation quality degrades — flatten or split instead
- You need the model to reason at length before answering and cannot afford a reasoning field
Tradeoffs
GainParse failures approach zero, and downstream code can trust its inputs
CostRigid schemas can constrain a model into forcing bad data into required fields
GainThe schema doubles as documentation and as a validation contract in tests
CostSchema changes are deployments; a new field requires coordinated rollout
GainEnums make invalid categorical values structurally impossible
CostConstrained decoding is not supported identically across providers and models
The shift is from treating the model as a text generator you post-process into treating it as a typed function call. Once the output type is declared, most of the defensive parsing code in a typical LLM pipeline simply disappears.
Declare the type, derive the schema
Write the schema once in your language's type system and generate the JSON Schema from it. Two things pay for themselves immediately: field descriptions, which the model reads and treats as instructions, and constraints — enums, ranges, lengths — which the decoder enforces mechanically.
class LineItem(BaseModel):
description: str
quantity: int = Field(ge=1, description="Units billed. Never zero.")
unit_price_cents: int = Field(ge=0, description="Integer cents. Never a float.")
class Invoice(BaseModel):
vendor_name: str
invoice_number: str = Field(description="Exactly as printed, including any prefix.")
issued_on: date
currency: Literal["USD", "EUR", "GBP", "INR"]
line_items: list[LineItem] = Field(min_length=1)
total_cents: int = Field(ge=0, description="Must equal the sum of line items.")
response = client.messages.create(
model=MODEL,
messages=[{"role": "user", "content": document}],
tools=[{"name": "record_invoice", "input_schema": Invoice.model_json_schema()}],
tool_choice={"type": "tool", "name": "record_invoice"}, # force the shape
)
invoice = Invoice.model_validate(response.content[0].input)Money as integer cents, never floats. Ask for a float and you will eventually receive 19.989999999999998, or a string, or a value with a currency symbol embedded. An integer field with a description saying "integer cents" removes the whole class of problem.
Leave room to think
Constrained decoding forces the first token into the schema, which removes the model's ability to reason before committing — and on hard extractions that measurably lowers accuracy. Give it somewhere to think inside the schema: a reasoning field ordered before the answer fields. Because JSON generation is sequential, the model writes its reasoning first and conditions the answer on it.
class Classification(BaseModel):
# Field ORDER matters: this is generated first and conditions what follows.
reasoning: str = Field(description="Brief analysis before deciding. 1-3 sentences.")
category: Literal["bug", "feature_request", "question", "billing", "other"]
confidence: float = Field(ge=0, le=1)Model absence explicitly
A required field the source does not contain forces the model to invent something — this is the single largest cause of hallucinated structured data. Make optional things genuinely optional, or better, model absence as a value the model can choose. `total_cents: int | None` with a description saying to use null when no total is printed produces null; `total_cents: int` produces a plausible fabricated number.
- Derive JSON Schema from your types; never hand-maintain both
- Write field descriptions as instructions — the model reads them
- Put a reasoning field first when the task needs thought before commitment
- Make absent data expressible as null; a required field will be fabricated
- Use enums for every closed set, and integers for money
A schema guarantees shape, not truth. `total_cents: 45000` is schema-valid whether or not the invoice says $450. Cross-field validation — does the total equal the sum of line items — is your job, and it catches errors constrained decoding cannot.