---
title: "Agentic Manual Testing"
description: "Directing coding agents to actively exercise running servers, APIs, CLI one-liners, and browser automation to verify real system behaviour."
section: "concepts"
type: "concept"
created: "2026-08-22"
updated: "2026-08-22"
confidence: "high"
tags: ["technique", "agents", "feedback-loops", "workflow"]
canonical: "https://pyweb.dev/wiki/agentic-manual-testing"
---
# Agentic Manual Testing

Automated unit tests are necessary, but code that passes unit tests can still fail to boot, miss styling, or crash on live input. Agentic manual testing is the discipline of having the agent spin up the software and actively interact with it before declaring completion.

## Mechanisms
- **CLI / One-liners:** Direct the agent to execute edge cases directly via `python -c "..."` or compile temporary test scripts in `/tmp`.
- **API Exploration:** Have the agent start a local development server (e.g. `python -m http.server` or app backend) and explore JSON endpoints with `curl`.
- **Headless Browser Automation:** Direct agents using browser tools (Playwright, Puppeteer, agent-browser, or CLI wrappers like `rodney`) to click buttons, fill forms, verify layouts, and check console errors.
- **Evidence Capture:** Require verifiable outputs (terminal logs, rendered HTML, or tool execution transcripts) rather than accepting self-reported completion.

## "First Run The Tests"
Starting a coding session with a prompt like `"First run the tests"` or `"Run uv run pytest"` forces the agent to discover test commands, gauges codebase complexity, and anchors the agent in a verification mindset from turn one.

## Related
- [simon willison](/wiki/simon-willison)
- [agentic engineering patterns](/wiki/agentic-engineering-patterns)
- [red green tdd](/wiki/red-green-tdd)
- [tdd with agents](/wiki/tdd-with-agents)
- [karpathy four guidelines](/wiki/karpathy-four-guidelines)
- [closed loop agent improvement](/wiki/closed-loop-agent-improvement)
- [automated eval engineering](/wiki/automated-eval-engineering)

---

## Agent Navigation

cluster: person (170 pages) | betweenness: 100.9

### References (outbound)
- [Simon Willison](https://pyweb.dev/wiki/simon-willison.md)
- [Agentic Engineering Patterns](https://pyweb.dev/wiki/agentic-engineering-patterns.md)
- [Red/Green TDD](https://pyweb.dev/wiki/red-green-tdd.md)
- [TDD With Agents](https://pyweb.dev/wiki/tdd-with-agents.md)
- [Karpathy's Four Guidelines](https://pyweb.dev/wiki/karpathy-four-guidelines.md)
- [Closed-Loop Agent Improvement](https://pyweb.dev/wiki/closed-loop-agent-improvement.md)
- [Automated Eval Engineering](https://pyweb.dev/wiki/automated-eval-engineering.md)

### Referenced by (inbound)
- [Automated Eval Engineering](https://pyweb.dev/wiki/automated-eval-engineering.md)
- [Closed-Loop Agent Improvement](https://pyweb.dev/wiki/closed-loop-agent-improvement.md)
- [Designing for Verifiability](https://pyweb.dev/wiki/designing-for-verifiability.md)
- [Generator-Evaluator Loop](https://pyweb.dev/wiki/generator-evaluator-loop.md)
- [Red/Green TDD](https://pyweb.dev/wiki/red-green-tdd.md)
- [Simon Willison](https://pyweb.dev/wiki/simon-willison.md)

### Evidence (verified primary sources)
- [simon-willison-agentic-manual-testing-2026](https://pyweb.dev/wiki/raw/articles/simon-willison-agentic-manual-testing-2026.md) | origin: https://simonwillison.net/guides/agentic-engineering-patterns/agentic-manual-testing/ | ingested: 2026-08-22 | sha256: 20f0c6dcf421283fa8a4b4e7804e359862a00a012c1e7db90fb1598893de848b
- [simon-willison-first-run-the-tests-2026](https://pyweb.dev/wiki/raw/articles/simon-willison-first-run-the-tests-2026.md) | origin: https://simonwillison.net/guides/agentic-engineering-patterns/first-run-the-tests/ | ingested: 2026-08-22 | sha256: 047d4e368e159e26eab8c43d7af8f48ade68354aad009fa1e86536bb4da10af4

### Machine endpoints
- Knowledge graph: https://pyweb.dev/api/graph.json
- Graph analysis: https://pyweb.dev/api/graph-analysis.json
- Context index: https://pyweb.dev/llms.txt