loading…

Structured Outputs

Often the thing you want back from an LLM is not text but an object: extracting multiple properties from unstructured input (PDFs, comments, documents) is one of the most powerful and business-relevant LLM use cases, alongside classification into enums/categories.

{
  "name": "verdict",
  "strict": true,
  "schema": {
    "type": "object",
    "properties": { "pass": {"type": "boolean"}, "reason": {"type": "string"} },
    "required": ["pass", "reason"],
    "additionalProperties": false
  }
}

Variants

• Object generation: pass a JSON schema, receive a typed object (data extraction from PDFs and other unstructured sources).
• Enum generation: constrain output to a fixed set of enumerated values (classification, sentiment, routing).
• Array generation: schema with multiple items for batch extraction.
• Streaming objects: instead of waiting for the whole object, stream it field-by-field as generation proceeds.
• Tool-based structuring: reuse tool calling loop infrastructure — declare a tool whose arguments are your schema and force the model to call it — to get the same shape guarantees without a dedicated structured-output API.

Why It Matters

Structured outputs are the bridge between probabilistic text generation and deterministic application code: they make LLM output consumable by databases, validators, and pipelines. The pattern pairs naturally with evals skills — schema conformance is cheap to verify automatically, a rare machine-checkable oracle.

Failure Modes

Symptom	Root cause	Fix
Schema violated at runtime	Model improvises fields	Strict schema mode + validation retry on parse failure
Over-constrained schema chokes output	Too many required fields	Required only what downstream code reads; everything else optional
Silent parse fallback	JSON extracted with regex	Fail loudly on malformed output; route to retry, never to guess

Rule of Thumb

Schema conformance is the cheapest deterministic oracle available - prefer it over any judge when the property is checkable.

Related

llm message protocol, tool calling loop, model provider abstraction, evals skills, generator evaluator loop.