First run the tests

Simon Willison’s Weblog Subscribe Sponsored by: Teleport — AI agents don’t sleep and will try anything to achieve their goal. Teleport explains how to deploy AI safely, starting with an isolated ephemeral trusted runtime.

Guides > Agentic Engineering Patterns

First run the tests

Automated tests are no longer optional when working with coding agents.

The old excuses for not writing them - that they’re time consuming and expensive to constantly rewrite while a codebase is rapidly evolving - no longer hold when an agent can knock them into shape in just a few minutes.

They’re also vital for ensuring AI-generated code does what it claims to do. If the code has never been executed it’s pure luck if it actually works when deployed to production.

Tests are also a great tool to help get an agent up to speed with an existing codebase. Watch what happens when you ask Claude Code or similar about an existing feature - the chances are high that they’ll find and read the relevant tests.

Agents are already biased towards testing, but the presence of an existing test suite will almost certainly push the agent into testing new changes that it makes.

Any time I start a new session with an agent against an existing project I’ll start by prompting a variant of the following:

For my Python projects I have pyproject.toml set up such that I can prompt this instead: These four word prompts serve several purposes:

It tells the agent that there is a test suite and forces it to figure out how to run the tests. This makes it almost certain that the agent will run the tests in the future to ensure it didn’t break anything. Most test harnesses will give the agent a rough indication of how many tests they are. This can act as a proxy for how large and complex the project is, and also hints that the agent should search the tests themselves if they want to learn more. It puts the agent in a testing mindset. Having run the tests it’s natural for it to then expand them with its own tests later on.

Similar to “Use red/green TDD”, “First run the tests” provides a four word prompt that encompasses a substantial amount of software engineering discipline that’s already baked into the models.

← Red/green TDD Agentic manual testing →

This is a chapter from the guide Agentic Engineering Patterns.

Chapters in this guide

Principles What is agentic engineering? Writing code is cheap now Hoard things you know how to do AI should help us produce better code Anti-patterns: things to avoid Working with coding agents How coding agents work Using Git with coding agents Subagents Testing and QA Red/green TDD First run the tests Agentic manual testing Understanding code Linear walkthroughs Interactive explanations Annotated prompts GIF optimization tool using WebAssembly and Gifsicle Adding a new content type to my blog-to-newsletter tool Appendix Prompts I use testing 93 tdd 5 ai 2,198 llms 1,914 coding-agents 240 ai-assisted-programming 402 generative-ai 1,947 agentic-engineering 59

Created: 24th February 2026 Last modified: 28th February 2026 4 changes

Previous: Red/green TDD

Next: Agentic manual testing