--- title: "Agent Design Is Still Hard" section: "raw" type: "source" created: "2026-08-27" updated: "2026-08-27" canonical: "https://pyweb.dev/wiki/raw/articles/armin-ronacher-agent-design-is-still-hard-2025" --- # Agent Design Is Still Hard By Armin Ronacher (November 21, 2025). Originally published on *Armin Ronacher's Thoughts and Writings* (lucumr.pocoo.org). A critical practitioner analysis of agent harness engineering: SDK abstraction leakage, explicit caching mechanics, reinforcement loops, failure isolation, and the evaluation bottleneck. ## Core Extraction Summary ### 1. Named Frameworks & Patterns (Author's Exact Words) - **"SDK Abstraction Leakage"**: Higher-level meta-SDKs (e.g., Vercel AI SDK) failing at complex agent tasks due to model-specific tool calling, cache control, and message format mismatches. - **"Explicit Cache Management"**: Manually controlling Anthropic cache breakpoints (system prompt + conversation head + rolling conversation tail) rather than relying on automatic caching. - **"Reinforcement in the Agent Loop"**: Injecting objective reminders, tool recovery hints, and background state changes into tool return payloads to prevent context drift. - **"Self-reinforcement tools"** (e.g. Claude Code todo tool): Tools that echo back an agent's task list to anchor focus across long trajectories. - **"Failure Isolation via Subagents"**: Running error-prone iterative tasks in isolated subagents so failed execution traces and bloated error contexts do not pollute the main context window. - **"Shared Virtual Filesystem"**: Using a common filesystem layer as the universal state and data conduit between heterogeneous tools (e.g. code execution, image generation, inference) to prevent dead-end tools. - **"Output Tool with Reinforcement Forcing"**: Separating internal agent execution traces from external human communication via an explicit output tool, reinforced if the agent attempts to exit without calling it. - **"Observability-Based Evals"**: Instrumenting live execution traces and test runs rather than evaluating prompts against static external benchmarks. ### 2. Decision Rules - **When building an agent harness**, target provider SDKs directly rather than high-level multi-provider meta-SDKs, **because** subtle model differences in tool formats, cache behavior, and error handling break generic abstractions. - **When designing tool responses**, do not just return raw data; inject contextual reinforcement (task status, next-step hints), **because** reinforcement keeps the model anchored on the objective. - **When executing failure-prone code tasks**, isolate them inside a subagent, **because** isolating failures preserves main context tokens and cache validity while returning only actionable summaries. - **When designing multimodal agent tools**, integrate all tools around a shared virtual filesystem, **because** direct tool-to-tool conduits create dead ends. ### 3. Anti-Patterns & Failure Mechanisms - **"Dead-End Tools"**: Creating specialized tools whose output can only be consumed by one other tool rather than being accessible to the entire agent harness via a filesystem. - **"Sub-Model Tone Rewriting"**: Passing agent outputs through a secondary small LLM to adjust tone, which increases latency, reduces quality, and risks leaking intermediate reasoning. - **"Context Editing Cache Invalidation"**: Editing message history to remove failure traces, which inadvertently trashes prompt cache breakpoints and increases token costs. - **"Static Prompt Evals"**: Evaluating agent systems using prompt benchmarks rather than end-to-end execution observability traces. ### 4. Quantitative Claims & Qualifiers - Explicit cache architecture: placing cache points after the system prompt, at the conversation head, and a rolling cache point at the conversation tail stabilizes token economics. - "Testing and evals is the single hardest problem" in agent engineering; existing off-the-shelf eval platforms fail to handle multi-step agentic execution traces. ### 5. What the Source Does NOT Claim - Does **NOT** claim that agent design has converged on standard abstractions; explicitly argues that agent architecture is currently immature and requires hand-crafted loops. --- ## Full Text I felt like it might be a good time to write about some new things I’ve learned. Most of this is going to be about building agents, with a little bit about using agentic coding tools. **TL;DR**: Building agents is still messy. SDK abstractions break once you hit real tool use. Caching works better when you manage it yourself, but differs between models. Reinforcement ends up doing more heavy lifting than expected, and failures need strict isolation to avoid derailing the loop. Shared state via a file-system-like layer is an important building block. Output tooling is surprisingly tricky, and model choice still depends on the task. ### Which Agent SDK to Target? When building an agent, you can target underlying provider SDKs (OpenAI, Anthropic) or higher-level abstractions (Vercel AI SDK, Pydantic). We chose Vercel AI SDK provider abstractions and drove the loop ourselves—and we would not make that choice again. 1. Model differences are significant enough that generic SDK abstractions fail (cache control, reinforcement requirements, tool formatting). Targeting provider SDKs keeps you in control. 2. Provider-side tools (like Anthropic web search) create message formatting conflicts with meta-SDKs. ### Caching Lessons Anthropic's explicit cache management provides far greater cost predictability. Our architecture: - One cache point after the system prompt. - Two cache points at the conversation start, with the last moving with the conversation tail. - Dynamic info (like current time) is fed later to avoid trashing the cache. ### Reinforcement in the Agent Loop Tool responses should not merely return data; they should inject reinforcement: - Reminding the agent of overall objectives. - Providing hints on how to recover when a tool fails. - Informing about background state changes. - *Self-reinforcement*: Claude Code's todo write tool echoes task lists back to keep focus. ### Isolate Failures Hiding execution failures from the main context: 1. **Subagents**: Run exploratory/iterative tasks in a subagent; report back only success and a brief summary of failed approaches. 2. **Context editing**: Removing failure tokens from history (though this invalidates cache points). ### Shared Filesystem Agents based on code execution require a common virtual filesystem. This avoids "dead ends" where an image generation tool cannot pass output to a code execution tool for zip compression. ### Testing and Evals We find testing and evals to be the single hardest problem. Unlike static prompts, agent evals require full observability data and instrumentation of live test runs. No current off-the-shelf solution is fully satisfactory. --- ## Agent Navigation ### Machine endpoints - Knowledge graph: https://pyweb.dev/api/graph.json - Graph analysis: https://pyweb.dev/api/graph-analysis.json - Context index: https://pyweb.dev/llms.txt