---
title: "LLM App Improvement Ladder"
description: "Matt Pocock's ordered ladder of 17 techniques for improving an LLM-powered app — from prompt basics to fine-tuning — try the simple thing first."
section: "concepts"
type: "concept"
created: "2026-08-30"
updated: "2026-08-30"
confidence: "high"
tags: ["llm-fundamentals", "patterns", "quality"]
canonical: "https://pyweb.dev/wiki/llm-app-improvement-ladder"
---
# LLM App Improvement Ladder

When an LLM app underperforms, don't jump to fine-tuning. Pocock's ordered ladder — try the simple thing first — moves from cheap prompt-level fixes to expensive architectural ones:^[raw/aihero/how-to-improve-your-llm-powered-app.md]

## The Rungs
1. **First prompt** — the baseline; most failures are under-specified instructions.
2. **Role-based prompting** — give the model a persona/role.
3. **XML tags** — structure the input so sections are unambiguous.
4. **Structured outputs** — schema-constrain the response ([structured outputs](/wiki/structured-outputs)).
5. **Reasoning** — elicit step-by-step thinking (cf. [think tool](/wiki/think-tool)).
6. **Multishot prompting** — examples in-prompt.
7. **Temperature** — tune randomness to the task.
8. **Tool calling** — let the model act ([tool calling loop](/wiki/tool-calling-loop)).
9. **LLM call chaining** — decompose into a pipeline (workflow pattern, [agents vs workflows](/wiki/agents-vs-workflows)).
10. **RAG** — retrieve documents into context.
11. **Chunking** — tune retrieval granularity.
12. **Agentic loops** — iterate tools against environment feedback.
13. **Parallelizing LLM calls** — fan out independent work.
14. **Evaluator-optimizer** — generate/evaluate loop ([generator evaluator loop](/wiki/generator-evaluator-loop)).
15. **LLM routers** — classify inputs to specialized handlers.
16. **Fine-tuning** — the last rung, not the first.

## Why the Order Matters
Each rung's cost compounds: prompt fixes are free, structural patterns cost engineering time, fine-tuning costs data plus pipeline maintenance. The ladder is also the diagnostic sequence — an app that fails at rung 3 won't be saved by rung 16. Measured by [eval taxonomy](/wiki/eval-taxonomy) at every step, or you're climbing blind.

## Related
[eval taxonomy](/wiki/eval-taxonomy), [structured outputs](/wiki/structured-outputs), [tool calling loop](/wiki/tool-calling-loop), [agents vs workflows](/wiki/agents-vs-workflows), [generator evaluator loop](/wiki/generator-evaluator-loop), [smart zone](/wiki/smart-zone).

---

## Agent Navigation

cluster: llm-fundamentals (8 pages) | betweenness: 40.3

### References (outbound)
- [Structured Outputs](https://pyweb.dev/wiki/structured-outputs.md)
- [Think Tool](https://pyweb.dev/wiki/think-tool.md)
- [Tool Calling Loop](https://pyweb.dev/wiki/tool-calling-loop.md)
- [Agents vs Workflows](https://pyweb.dev/wiki/agents-vs-workflows.md)
- [Generator-Evaluator Loop](https://pyweb.dev/wiki/generator-evaluator-loop.md)
- [Eval Taxonomy](https://pyweb.dev/wiki/eval-taxonomy.md)

### Referenced by (inbound)
- [LLM Message Protocol](https://pyweb.dev/wiki/llm-message-protocol.md)

### Evidence (verified primary sources)
- raw/aihero/how-to-improve-your-llm-powered-app.md (internal workspace doc)

### Machine endpoints
- Knowledge graph: https://pyweb.dev/api/graph.json
- Graph analysis: https://pyweb.dev/api/graph-analysis.json
- Context index: https://pyweb.dev/llms.txt