Skills-Native Evaluation and Agent Adversarial Training

Two breakthrough patterns are reshaping how we build reliable AI agents: evaluation methodology packaged as portable skills, and adversarial feedback loops that separate code generation from quality assessment. These advances move beyond prompt engineering toward systematic harness design and verifiable improvement cycles.

Evaluation as Agent Skills, Not Tools

Traditional evaluation tools push teams toward generic metrics before they understand their data. Hamel Husain and Shreya Shankar flipped this with Evals Skills — evaluation methodology shipped as installable agent skills that route teams through proven workflows based on their specific situation.

Instead of another dashboard or framework, the methodology becomes procedural instructions a coding agent loads on demand. The entry skill start analyzes your setup and routes to specialized workflows: eval-audit for existing pipelines, error-discovery for unanalyzed traces, and domain-specific skills like evaluate-rag for retrieval systems.

This exemplifies progressive disclosure — giving agents just enough context to proceed effectively without overwhelming the smart zone with comprehensive documentation. Each skill encodes hard-won lessons “from helping 50+ companies” into executable procedures that avoid “many easily avoidable footguns.”

The approach acknowledges a core truth about error analysis and evals: real evaluation requires human domain expertise to label failure modes, not automated scoring against generic rubrics. As Husain puts it, the revenge of the data scientist means “every recurring eval pitfall maps to a missing data-science fundamental.”

Adversarial Agent Training

Meanwhile, Anthropic’s Prithvi Rajasekaran tackled a different reliability problem: agents confidently praising their own mediocre work. His solution borrows from Generative Adversarial Networks, creating a generator-evaluator loop that separates the agent doing work from the agent judging it.

For frontend design — where there’s “no binary check equivalent to a verifiable software test” — the harness converts subjective quality into four gradable criteria: design quality, originality, craft, and functionality. The evaluator uses Playwright to navigate live pages, screenshot implementations, and score against explicit criteria that “penalized highly generic ‘AI slop’ patterns” like “purple gradients over white cards.”

The key insight: “tuning a standalone evaluator to be skeptical turns out to be far more tractable than making a generator critical of its own work.” Runs lasted 5-15 iterations over four hours, with even first iterations beating unprompted baselines before any adversarial feedback.

Scaling to Multi-Agent Systems

The pattern scales to full-stack development through a three-agent architecture: planner expands simple prompts into detailed specs, generator works one feature per sprint, and evaluator exercises the running application via browser automation.

Before each sprint, generator and evaluator negotiate a sprint contract — agreeing on what “done” looked like before any code was written. Sprint 3 alone had 27 criteria covering the level editor. This creates verifiable success criteria rather than vague user stories.

The economics are stark: a solo agent built a game in 20 minutes for $9, but the full harness took 6 hours for $200 — “over 20x more expensive, but the difference in output quality was immediately apparent.” The solo run’s game was broken; the harness build actually worked.

The Context Reset Innovation

Rajasekaran’s work also advances our understanding of context rot. While compaction summarizes earlier conversation to fit context limits, context resets start fresh agents with structured handoffs. Resets prevent “context anxiety” — models wrapping up prematurely as they approach perceived limits — which compaction can’t fix because “it doesn’t give the agent a clean slate.”

This connects to broader patterns in agent harness engineering where durable session state and clean handoff artifacts enable multi-session work that surpasses single-context capabilities.

Infrastructure for Reliable Agents

Both developments point toward the same architectural shift: moving from prompt tweaks to systematic harness design. Whether evaluation methodology distributed as skills or adversarial training loops, the focus shifts to the substrate around the model — execution environment, feedback loops, and verification gates — as the primary driver of reliability.

This fits Drew Breunig’s economic observation that once frontier capability stopped arriving at flat prices, teams “started to think about what work went where.” Harness and evaluation investment stopped being throwaway glue and became durable competitive advantages.

The convergence suggests a maturing field: instead of hoping better prompts will solve reliability, we’re building the infrastructure that makes agent outputs verifiable, improvable, and trustworthy at scale.