---
title: "Human Judgment Doesn't Leave the Software Factory. It Relocates."
section: "raw"
type: "source"
created: "2026-08-27"
updated: "2026-08-27"
canonical: "https://pyweb.dev/wiki/raw/articles/addy-osmani-human-judgment-software-factory-2026"
---
# Human Judgment Doesn't Leave the Software Factory. It Relocates.

By Addy Osmani (2026). Originally published on Substack.

An examination of repeatable software factories, lights-on operations, and where human taste and judgment must be placed.

## Core Extraction Summary

### 1. Named Frameworks & Patterns (Author's Exact Words)
- **"Software Factory"**: A repeatable, event-driven loop around software engineering tasks (triage, implementation, verification, and monitoring).
- **"Lights-on Software Factory"**: Operating model where autonomous agents execute background workflows under explicit human oversight, bounds, and quality constraints.
- **"Relocation of Human Judgment"**: The architectural principle that humans should be removed from mechanical inner execution steps and concentrated upstream (intent, system design, quality bar) and downstream (evidence evaluation, risk assessment, merge ownership).
- **"Triage Label State Machine"**: Work queue coordination pattern using explicit GitHub issue labels (`ready-to-implement`, `ready-to-spec`, `needs-info`, `wait-to-implement`) as both queue and concurrency lock.
- **"Steering, Notifications, Handoffs"**: The triad of human-agent interaction mechanisms inside a factory.

### 2. Decision Rules
- **When designing a software factory**, encode human quality taste into deterministic environment constraints (linters, type systems, mutation tests, architectural boundaries), **because** human attention cannot scale to read every diff.
- **When an agent encounters ambiguity or high-risk operations**, configure the factory to halt and emit a notification for human steering, **because** agents will otherwise extrapolate intent with confident hallucinations.
- **When coordinating multiple parallel agent sessions**, use explicit triage labels and progress files to prevent sessions from claiming overlapping tasks.

### 3. Anti-Patterns & Failure Mechanisms
- **"Lights-out factory fantasy"**: Attempting to run unmonitored end-to-end software development without human boundary gates.
- **"Context compaction amnesia"**: Long agent sessions losing architectural rationale due to context compaction, leaving behind code whose decision history is unknown.
- **"Number of checks != quality"**: Piling on dozens of noisy checks without tuning signal-to-noise ratio, causing alert fatigue.

### 4. Quantitative Claims & Qualifiers
- Osmani routinely manages 5–10 concurrent agent sessions across production client applications, open-source repos, and experimental tools.
- Operating multiple parallel sessions magnifies comprehension debt unless session trajectories and decisions are explicitly recorded to disk.

### 5. What the Source Does NOT Claim
- Does **NOT** claim software factories are necessary for every project; explicitly notes that individual engineers get surprisingly far using stock coding harnesses (Claude Code, Codex) with good specs and verification commands.

---

## Full Text

A software factory is a repeatable loop around software work. If you’re building a software factory, code good enough to ship still needs human taste and ownership.

- You’ll need humans in the loop upfront for deciding product intent, system design, and your quality bar.
- Review code (lights-on factory) where automated back-pressure breaks or maintainability trade-offs arise.
- Aim for quality checks to happen early and continuously (types, tests, mutation testing, security scanning, architecture linters).
- Number of checks != quality. Experiment with checks to maximize signal-to-noise ratio.

### Do You Really Need a Software Factory?

In my experience, you can get surprisingly far with your stock coding harness (Claude Code, Codex, multiple sessions, good specs, constraints). A software factory becomes useful when work must be **repeatable and event-driven** across an isolated cloud queue.

### Factory Interaction Triad

- **Steering**: Course-correcting an agent when it goes off track.
- **Notifications**: How the factory signals that a requirement is ambiguous or risky.
- **Handoff**: Moving context, state, and remaining tasks between cloud agents and human reviewers.

### Cognitive Bandwidth and Comprehension Debt

Human cognitive bandwidth does not scale linearly with parallel agents. Running 5–10 sessions creates multiple cold mental models. Asking agents to store their trajectory and lessons learned allows engineers to consult decisions later without suffering comprehension debt.

### Ownership Doesn't Disappear

The percentage of code physically typed by humans may fall dramatically. Human ownership does not fall with it. Someone still chooses the problem, architecture, quality bar, and verification signals. When the system breaks, "the agent wrote it" is not an excuse. Human judgment is being relocated.

---

## Agent Navigation

### Machine endpoints
- Knowledge graph: https://pyweb.dev/api/graph.json
- Graph analysis: https://pyweb.dev/api/graph-analysis.json
- Context index: https://pyweb.dev/llms.txt