---
title: "LLM Evals: Everything You Need to Know (AI Evals FAQ)"
section: "raw"
type: "source"
created: "2026-08-27"
updated: "2026-08-27"
canonical: "https://pyweb.dev/wiki/raw/articles/hamel-husain-ai-evals-faq-2026"
---
# LLM Evals: Everything You Need to Know (AI Evals FAQ)

By Hamel Husain (with contributions from Shreya Shankar).
Published at hamel.dev/blog/posts/evals-faq/ (2026).

## Executive Summary & Core Philosophy

Evaluation is not a separate testing phase or an optional line item; it is the core iterative engine of AI product development. Unsuccessful AI products almost always suffer from a single root failure: lack of robust, domain-calibrated evaluation systems. In contrast to traditional software engineering where debugging is episodic, AI engineering requires continuous error analysis of raw model outputs and traces.

## Named Frameworks & Patterns

1. **TPR/TNR Calibration:** Aligns LLM-as-a-judge against human ground truth on held-out datasets by optimizing True Positive Rate (sensitivity) and True Negative Rate (specificity) rather than aggregate accuracy or correlation coefficients.
2. **The 100% Pass Rate Failure Mode:** An evaluation suite showing a 100% pass rate is a diagnostic signal of an unchallenging, broken, or gamed evaluation. Meaningful evaluations maintain tension against the system boundary (e.g., target ~70% pass rates under stress).
3. **Transition Failure Matrices:** A diagnostic grid for multi-step agentic workflows where rows represent the last successful state and columns represent the first failure state (e.g., `GenSQL -> ExecSQL` vs `DecideTool -> PlanCal`), isolating error hotspots.
4. **Benevolent Dictator Domain Expert:** Concentrating quality arbitration in a single domain expert who understands user intent, avoiding multi-annotator committee noise and inter-annotator disagreement.
5. **Critique Shadowing & Binary Pass/Fail:** Requiring human and model judges to emit a binary pass/fail verdict paired with a concrete textual critique string, rather than 1-5 Likert scores.
6. **Error Analysis Flywheel:** The virtuous cycle linking trace inspection (Level 2), unit test assertions (Level 1), fine-tuning data curation, and targeted error remediation.

## Core Decision Rules

- **When to start evals:** Spend 30 minutes manually reviewing 20–50 traces in a notebook before building automated infrastructure, because understanding empirical failure modes must precede metric selection.
- **When choosing scoring scales:** Use binary pass/fail judgments with textual critiques instead of 1–5 Likert scales, because Likert scales introduce extreme human variance and make LLM-as-a-judge calibration nearly impossible.
- **When evaluating agentic workflows:** Separate evaluation into two distinct tiers:
  1. *End-to-end task success:* Black-box evaluation of whether the user's primary goal was achieved (exact answer, correct side-effect).
  2. *Step-level diagnostics:* Instrumented trace evaluation using transition failure matrices to pinpoint where intermediate state transitions broke.
- **When selecting judge models:** Start with the most capable frontier model to establish high TPR/TNR alignment against held-out human labels before attempting distillation or cost optimization.
- **When evaluating RAG vs Agent systems:** Evaluate retrieval and generation independently; never evaluate the generation step when retrieval has failed.

## Anti-Patterns & Failure Mechanisms

- **Generic Off-The-Shelf Metrics:** Using out-of-the-box vendor metrics (ROUGE, BLEU, generic toxicity/helpfulness rubrics) that have zero calibration against actual product domain failure modes.
- **Premature Automation:** Building automated LLM judges or complex CI dashboards before conducting manual trace inspection.
- **Committee Annotation Grids:** Averaging scores from multiple non-expert annotators with vague rubrics, producing noisy labels that cannot train or calibrate judges.
- **Pass-Rate Vanity:** Designing evaluations to pass cleanly rather than actively probing failure boundaries.

## Quantitative Claims & Qualifiers

- **60–80% of development time** in production AI projects is spent on manual error analysis and trace inspection rather than model training or prompt tweaking.
- **700+ engineers and product managers** trained across live AI Evals cohorts.
- **~70% pass rate** is a healthier target for active development benchmarks than 100%, indicating genuine stress-testing of system edge cases.

## What the Source Does NOT Claim

- Does **not** claim that LLM-as-a-judge can entirely replace human error analysis and domain oversight.
- Does **not** claim that foundation model benchmarks (MMLU, HELM, SWE-bench) predict product-level task reliability.
- Does **not** claim that 100% test pass rates represent high product quality.
- Does **not** claim that complex evaluation frameworks should precede simple notebook-based data inspection.

---

## Agent Navigation

### Machine endpoints
- Knowledge graph: https://pyweb.dev/api/graph.json
- Graph analysis: https://pyweb.dev/api/graph-analysis.json
- Context index: https://pyweb.dev/llms.txt