--- title: "Demystifying evals for AI agents" section: "raw" type: "source" created: "2026-08-27" updated: "2026-08-27" canonical: "https://pyweb.dev/wiki/raw/articles/anthropic-demystifying-evals-for-ai-agents-2026" --- # Demystifying evals for AI agents **Authors:** Mikaela Grace, Jeremy Hadfield, Rodrigo Olivares, Jiri De Jonghe (Anthropic Engineering) **Published:** 2026-01-09 **Source:** [Anthropic Engineering Blog](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) **Contributors & Collaborators:** Nicholas Carlini, David Hershey, Gian Segato, Mike Merrill, Alex Shaw, Ethan Dixon, Pedram Navid, Jake Eaton, Alyssa Baum, Lina Tawfik, Karen Zhou, Alexander Bricken, Sam Kennedy, Robert Ying; partners including Stripe, Cognition, Bolt, Sierra, Descript. --- ## Executive Summary & Core Thesis Good evaluations enable teams to ship AI agents with confidence. Without evals, agent development falls into reactive feedback loops: discovering bugs in production where fixing one regression silently introduces others. Evaluating AI agents is fundamentally harder than evaluating single-turn models. Agents operate over many turns—calling tools, modifying external environment state, and adapting to intermediate outputs. In agent systems, **mistakes compound non-linearly across turns**. Furthermore, evaluating an agent means evaluating the **agent harness (scaffold) and the model working together**. --- ## Formal Structural Model of Agent Evaluations - **Task (Problem / Test Case):** Single test unit with defined inputs, initial environment state, and success criteria. - **Trial:** Individual execution attempt on a task. Multiple trials are required to measure variance from non-deterministic generation and branching paths. - **Grader:** Scoring logic evaluating transcript or outcome assertions. Tasks typically employ multiple graders. - **Transcript (Trace / Trajectory):** Complete historical record of a trial, including all messages, tool calls, tool outputs, and reasoning steps. - **Outcome:** Final state of the environment at trial conclusion (e.g., verifying a database reservation record exists, rather than trusting an agent transcript asserting "flight booked"). - **Evaluation Harness:** Infrastructure that orchestrates tasks concurrently, provides mock environments, records trajectories, executes graders, and aggregates metrics. - **Agent Harness (Scaffold):** Runtime system enabling the model to act as an agent (e.g. Claude Code harness, Agent SDK primitives). - **Evaluation Suite:** Curated collection of tasks measuring targeted capabilities or behavioral guardrails. --- ## Grader Taxonomy & Trade-offs | Grader Type | Methods | Strengths | Weaknesses | | :--- | :--- | :--- | :--- | | **Code-based** | String checks, binary tests (fail-to-pass / pass-to-pass), static analysis, outcome state verification, tool call parameter assertions | Fast, inexpensive, 100% deterministic, objective, easy to debug | Brittle to valid semantic variations; cannot evaluate open-ended or subjective qualities | | **Model-based** | Rubric scoring, natural language assertions, pairwise comparison, reference comparison, multi-judge consensus | Highly flexible, scalable, captures nuance, handles freeform responses | Non-deterministic, token-expensive, susceptible to bias, requires human calibration | | **Human** | Domain expert review, golden set annotation, disagreement adjudication | Gold standard for truth, handles high-stakes safety and edge nuance | Expensive, slow, unscalable for continuous CI/CD | --- ## Lifecycle Evolution of Agent Evals (Claude Code Case Study) 1. **Stage 1 (Prototyping):** Fast dogfooding, manual user feedback, and intuition. 2. **Stage 2 (Narrow Targeted Evals):** Automated evals for isolated, high-leverage failure modes (e.g. prompt concision, precise file editing diffs). 3. **Stage 3 (Complex Behavioral Evals):** Multi-turn behavioral evals for subtle failure modes (e.g. over-engineering, unnecessary refactoring, context compaction survival). 4. **Stage 4 (Production & A/B Evals):** Continuous telemetry, A/B testing, regression testing on static task banks. --- ## Named Frameworks & Patterns (Verbatim) - **"Scaffold-and-model joint evaluation thesis"**: An evaluation scores the agent harness and model together; harness improvements alter benchmark scores as significantly as model weight upgrades. - **"Outcome verification vs transcript assertion"**: Verifying environmental ground truth (SQL state, git commits, passing test suites) rather than parsing the agent's textual assertions. - **"Multi-turn error compounding"**: The compounding probability of failure as agents take multi-step branching tool actions. - **"Creative loophole failure mode"**: Frontier models discovering valid policy loopholes (e.g. Opus 4.5 solving tau2-bench booking tasks via unpredicted policy exceptions) that fail rigid static grading assertions. --- ## Decision Rules - **When evaluating an AI agent, do measure final environment outcomes (e.g., SQL records, file system state) rather than transcript assertions**, because agents can assert success while failing to alter environment state. - **When benchmarking agent performance, do execute multiple trials per task**, because LLM non-determinism and multi-turn branching produce variance across runs. - **When designing graders, do layer code-based deterministic checks first before using model-based rubrics**, because code graders are fast, reproducible, and cheap while model graders add non-determinism. - **When evolving agent harnesses (e.g. Claude Code), do start with narrow targeted evals (conciseness, file edits) before progressing to complex behavioral evals (over-engineering)**, because baseline harness stability must precede complex behavioral tuning. --- ## Anti-Patterns & Failure Modes (with Mechanisms) ### 1. Transcript-Only Self-Reporting - **Mechanism:** Graders asserting success by parsing final LLM text output ("I fixed the bug") rather than running environment tests or verifying modified code. - **Observable Symptom:** Agent hallucinates success, reports positive completion, but leaves broken code or empty database tables. ### 2. Single-Turn Benchmark Extrapolation - **Mechanism:** Assuming that benchmark scores on static, single-turn benchmarks (e.g. MMLU, standard coding puzzles) predict multi-turn agentic performance. - **Observable Symptom:** Models with high single-turn capability scores fail catastrophically in multi-turn environments due to lack of tool orchestration and state recovery capabilities. --- ## What the Source Does NOT Claim - Does **NOT** claim that automated evaluations eliminate the necessity of human domain expert spot-checks. - Does **NOT** claim that single-turn benchmark metrics predict multi-turn agent task completion rates. --- ## Agent Navigation ### Machine endpoints - Knowledge graph: https://pyweb.dev/api/graph.json - Graph analysis: https://pyweb.dev/api/graph-analysis.json - Context index: https://pyweb.dev/llms.txt