Structured Outputs and JSON Mode

Most production applications need more than free-form text — they need the model's response to be parseable by code. Structured output techniques constrain the model to emit valid JSON (or another format) that your application can reliably consume.

JSON mode vs. schema-constrained output

JSON mode (available on most providers) guarantees valid JSON syntax but does not enforce a specific schema — the keys and values are still up to the model. Schema-constrained output (OpenAI's response_format with json_schema, or Anthropic's tool-use pattern) enforces both syntax and shape, making downstream parsing far more robust.

from pydantic import BaseModel
from openai import OpenAI

client = OpenAI()

class Sentiment(BaseModel):
    label: str   # "positive" | "negative" | "neutral"
    score: float  # 0.0 to 1.0
    reasoning: str

completion = client.beta.chat.completions.parse(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Review: Great product, fast shipping!"}],
    response_format=Sentiment,
)
result: Sentiment = completion.choices[0].message.parsed
print(result.label, result.score)

Function calling for tool use and extraction

Function calling lets you describe a function signature to the model. Instead of returning prose, the model emits a structured call with arguments matching the schema. This pattern works for both data extraction and for routing the model's intent to actual code.

Validation and fallback

Even with schema constraints, validate parsed output before using it. Check that enum values are expected, numeric ranges are sensible, and required fields are present. When validation fails, retry with an error message injected as an assistant turn, asking the model to fix the specific problem.

• Use Pydantic (Python) or Zod (TypeScript) to define and validate schemas
• Retry at most 2 times on schema violation before falling back to a safe default
• Log every validation failure — patterns reveal prompt or schema design problems
• Keep schemas flat and simple; deeply nested structures increase parsing failures