---
title: "Agentic Code Quality"
description: "Risk-conditioned verification and authorization controls for autonomous software development."
section: "concepts"
type: "concept"
created: "2026-08-27"
updated: "2026-08-27"
confidence: "medium"
tags: ["coding-guidelines", "feedback-loops", "tdd", "workflow", "evaluation", "security"]
canonical: "https://pyweb.dev/wiki/agentic-code-quality"
---
# Agentic Code Quality

Agentic code quality is usefully modeled as a property of the **model–scaffold–toolchain–policy system**, not the generator alone. While practitioners emphasize patterns like [agentic engineering patterns](/wiki/agentic-engineering-patterns) and [eval driven development](/wiki/eval-driven-development), empirical research reveals distinct failure modes: test-oracle defects, reward hacking, scope expansion, noisy automated review, and long-horizon degradation. [agentic quality evidence](/wiki/agentic-quality-evidence) separates observed evidence from the architecture proposed here.

## Quality Vector

A software factory should report independent acceptance dimensions plus their joint intersection:

| Dimension | Direct evidence | Proxy to distrust |
|---|---|---|
| Requirement correctness | Hidden behavioral tests plus expert adjudication | Visible tests alone |
| Regression safety | Full-suite, differential, property, or metamorphic checks | Changed tests only |
| Security | Executable abuse cases and confirmed findings | Scanner count |
| Architecture | Dependency contracts around stable public boundaries | Folder naming |
| Test integrity | Protected baselines and mutation survival | Raw coverage |
| Scope discipline | Audited actions against an authorization allowlist | Task completion |
| Reviewability | True findings per reviewer minute (TPR/TNR calibrated) | Comment volume |
| Operability | Canary health, rollback, defect survival | Merge success |
| Economics | Cost and elapsed time per releasable change | LOC or PR count |

At the factory exit, report `correctness_pass`, `regression_pass`, `security_pass`, `architecture_pass`, and `scope_pass`, plus their joint **releasable patch rate**. Critical security, authorization, and data-loss events remain separate tail outcomes.

## Risk-Conditioned Control Architecture

```mermaid
flowchart TD
    I[Human Intent and Authorization] --> S[Executable Spec and Contracts]
    S --> C[Context Assembly]
    C --> G[Generator in Isolated Worktree]
    G --> F[Fast Deterministic Checks]
    F -->|Fail: bounded repair| G
    F --> B[Behavior and Regression Oracles]
    B -->|Fail: bounded repair| G
    B --> A[Architecture and Security Contracts]
    A -->|Fail: bounded repair| G
    A --> R[Calibrated Independent Review]
    R -->|Actionable finding: bounded repair| G
    R --> H{Risk Gate}
    H -->|Low risk and observable proof| D[Canary Deployment]
    H -->|Ambiguity or high blast radius| U[Accountable Human Decision]
    D --> T[Telemetry and Rollback]
    T --> S
```

### Tier 0 — Authorization and Containment
- Declare allowed repositories, paths, services, secrets, and deployment targets.
- Mount hidden conformance suites and protected regression baselines read-only outside the generator worktree.
- Keep candidate test paths writable so the generator can use [red green tdd](/wiki/red-green-tdd); evaluate those tests independently.
- Isolate filesystem, processes, credentials, and network egress to bound blast radius.^[raw/papers/overeager-coding-agents-2026.md]
- Record tool actions through audit channels the agent cannot bypass.

### Tier 1 — Fast Inner Loop
Run after each coherent edit:
- **TypeScript:** `oxlint` for fast AST checks (see [deterministic lint gates](/wiki/deterministic-lint-gates) for the error-only gate pattern); `tsc --noEmit --incremental` for type compatibility; changed Vitest/Jest unit tests. [[source: oxlint-type-aware-linting-2026]](/wiki/raw/articles/oxlint-type-aware-linting-2026)
- **Python:** Ruff lint/format; Pyright or mypy under repo policy; changed-scope pytest.
Fast checks create cheap back-pressure by catching local syntax and type impossibilities.

### Tier 2 — Structural and Behavioral Contracts
Before review:
- Hidden acceptance tests derived from requirements; full regression tests for affected dependency cones.
- Property, differential, or metamorphic checks where suitable; API and schema contracts.
- Graph conformance: `dependency-cruiser` rules for JavaScript/TypeScript and `import-linter` layer contracts for Python. [[source: dependency-cruiser-rules-reference-2026]](/wiki/raw/articles/dependency-cruiser-rules-reference-2026) [[source: import-linter-layer-contracts-2026]](/wiki/raw/articles/import-linter-layer-contracts-2026)

### Tier 3 — Test Strength and Adversarial Verification
Used selectively for critical domain, parser, financial, cryptographic, or security logic:
- Mutation testing via StrykerJS or `mutmut` to measure assertion strength. [[source: strykerjs-configuration-2026]](/wiki/raw/articles/strykerjs-configuration-2026) [[source: mutmut-documentation-2026]](/wiki/raw/articles/mutmut-documentation-2026)
- Adversarial probes: malformed input fuzzing and hacker–fixer–solver loops against acceptance verifiers.^[raw/papers/adversarial-hacker-fixer-verifiers-2026.md]

### Tier 4 — Calibrated Independent Review
- Independent judge agents review diffs against specifications, non-goals, and security policies before seeing implementer rationale.
- Reviewer agents must be calibrated on expert-labeled sets, reporting true positive (TPR) and true negative rates (TNR) to avoid low-signal review noise.^[raw/papers/code-review-agents-empirical-study-2026.md] [[source: hamel-husain-shreya-shankar-evals-skills-2026]](/wiki/raw/articles/hamel-husain-shreya-shankar-evals-skills-2026)

### Tier 5 — Risk Gate and Production Feedback
Auto-merge only for pre-approved low-risk changes meeting observable criteria:
- Conformance suite passes; touched resources match allowlist; regression checks pass without exceptions.
- Canary monitoring and rollback triggers active. Ambiguous intent or high blast radius requires accountable human sign-off.

## Controls by Task Risk

| Task | Minimum control | Human role |
|---|---|---|
| Documentation or isolated formatting | T0–T1 plus link/build checks | Sample audit |
| Local bug fix with reproducer | T0–T2 plus protected regression | Review exceptions |
| Feature in one bounded module | T0–T2 plus API/property checks | Approve spec or boundary changes |
| Core domain, parser, financial, crypto, auth | T0–T3 with mutation/adversarial checks | Mandatory accountable review |
| Cross-module refactor | T0–T4 with dependency graph & characterization | Review architecture & migration |
| Platform or data migration | T0–T5 with compatibility matrix & canary | Approve rollout |
| Ambiguous product behavior | Prototype and clarify first | Decide intent; no auto-merge |

## Oracle Health

A green verifier can still be flawed. Epoch and OpenAI audits demonstrated SWE-bench task contamination, test over-specificity, and exploitable test harness bugs. [[source: epoch-swe-bench-verified-analysis-2025]](/wiki/raw/articles/epoch-swe-bench-verified-analysis-2025) [[source: openai-swe-bench-verified-audit-2026]](/wiki/raw/articles/openai-swe-bench-verified-audit-2026) Production harnesses must treat evaluation oracles as production software subject to regression suites, mutation checks, and [eval driven development](/wiki/eval-driven-development).

## What Remains Unverified

- The causal effect of the complete multi-tier stack on escaped production defects.
- Long-term maintenance cost under task-matched random assignment.
- Mutation testing's cost-effectiveness as an autonomous merge gate.
- The strongest null: frontier model capability, basic compiler/linter feedback, and human task selection may drive nearly all observed success; multi-tier scaffolding may add compute and false rejection without reducing escapes.

## Related
- [five debts of agentic engineering](/wiki/five-debts-of-agentic-engineering) — generative debt modes and control mappings
- [releasable patch rate](/wiki/releasable-patch-rate) — north-star factory metric
- [constraint layering](/wiki/constraint-layering) — allocating controls across prompt, tools, and sandboxes
- [skill treatment effect](/wiki/skill-treatment-effect) — empirical evaluation of procedural skills
- [agentic quality evidence](/wiki/agentic-quality-evidence) — empirical evidence and contradictions
- [eval driven development](/wiki/eval-driven-development) — error-discovery eval loop
- [agentic engineering patterns](/wiki/agentic-engineering-patterns) — disciplined practitioner patterns
- [agentic software factory](/wiki/agentic-software-factory) — operating model
- [agent containment and blast radius](/wiki/agent-containment-and-blast-radius) — authorization and damage bounds
- [designing for verifiability](/wiki/designing-for-verifiability) — inspectable product contracts
- [agentic code quality entity accounting](/wiki/agentic-code-quality-entity-accounting) — cycle 1 source-author dispositions
- [agentic code quality cycle 2 entity accounting](/wiki/agentic-code-quality-cycle-2-entity-accounting) — cycle 2 source-author dispositions

---

## Agent Navigation

cluster: person (170 pages) | betweenness: 7322

### References (outbound)
- [Agentic Engineering Patterns](https://pyweb.dev/wiki/agentic-engineering-patterns.md)
- [Red/Green TDD](https://pyweb.dev/wiki/red-green-tdd.md)
- [Deterministic Lint Gates](https://pyweb.dev/wiki/deterministic-lint-gates.md)
- [Eval-Driven Development](https://pyweb.dev/wiki/eval-driven-development.md)
- [Five Debts of Agentic Engineering](https://pyweb.dev/wiki/five-debts-of-agentic-engineering.md)
- [Releasable Patch Rate](https://pyweb.dev/wiki/releasable-patch-rate.md)
- [Constraint Layering](https://pyweb.dev/wiki/constraint-layering.md)
- [Skill Treatment Effect](https://pyweb.dev/wiki/skill-treatment-effect.md)
- [Agentic Code Quality — Evidence Map](https://pyweb.dev/wiki/agentic-quality-evidence.md)
- [Agentic Software Factory](https://pyweb.dev/wiki/agentic-software-factory.md)
- [Agent Containment and Blast Radius](https://pyweb.dev/wiki/agent-containment-and-blast-radius.md)
- [Designing for Verifiability](https://pyweb.dev/wiki/designing-for-verifiability.md)
- [Agentic Code Quality Evidence — Entity Accounting](https://pyweb.dev/wiki/agentic-code-quality-entity-accounting.md)
- [Agentic Code Quality Cycle 2 — Entity Accounting](https://pyweb.dev/wiki/agentic-code-quality-cycle-2-entity-accounting.md)

### Referenced by (inbound)
- [Agent Harness Engineering](https://pyweb.dev/wiki/agent-harness-engineering.md)
- [Agentic Engineering Patterns](https://pyweb.dev/wiki/agentic-engineering-patterns.md)
- [Agentic Software Factory](https://pyweb.dev/wiki/agentic-software-factory.md)
- [Deterministic Lint Gates](https://pyweb.dev/wiki/deterministic-lint-gates.md)
- [Eval-Driven Development](https://pyweb.dev/wiki/eval-driven-development.md)
- [Five Debts of Agentic Engineering](https://pyweb.dev/wiki/five-debts-of-agentic-engineering.md)
- [Releasable Patch Rate](https://pyweb.dev/wiki/releasable-patch-rate.md)
- [Site-as-Eval-Subject Grading](https://pyweb.dev/wiki/site-as-eval-subject-grading.md)
- [Addy Osmani](https://pyweb.dev/wiki/addy-osmani.md)
- [Aditi Raghunathan](https://pyweb.dev/wiki/aditi-raghunathan.md)
- [Ahmed E. Hassan](https://pyweb.dev/wiki/ahmed-e-hassan.md)
- [Airbnb](https://pyweb.dev/wiki/airbnb.md)
- [Andrei Botocan](https://pyweb.dev/wiki/andrei-botocan.md)
- [Beatrice Casey](https://pyweb.dev/wiki/beatrice-casey.md)
- [Beth Barnes](https://pyweb.dev/wiki/beth-barnes.md)
- [Brittany Reid](https://pyweb.dev/wiki/brittany-reid.md)
- [Christoph Csallner](https://pyweb.dev/wiki/christoph-csallner.md)
- [Christoph Nakazawa](https://pyweb.dev/wiki/christoph-nakazawa.md)
- [David Gros](https://pyweb.dev/wiki/david-gros.md)
- [David Rein](https://pyweb.dev/wiki/david-rein.md)
- [Dipayan Banik](https://pyweb.dev/wiki/dipayan-banik.md)
- [DORA](https://pyweb.dev/wiki/dora.md)
- [Dung Nguyen Manh](https://pyweb.dev/wiki/dung-nguyen-manh.md)
- [Eirini Kalliamvakou](https://pyweb.dev/wiki/eirini-kalliamvakou.md)
- [Epoch AI](https://pyweb.dev/wiki/epoch-ai.md)
- [Florian Brand](https://pyweb.dev/wiki/florian-brand.md)
- [Gelei Deng](https://pyweb.dev/wiki/gelei-deng.md)
- [Gergely Orosz](https://pyweb.dev/wiki/gergely-orosz.md)
- [GitHub](https://pyweb.dev/wiki/github.md)
- [Hajimu Iida](https://pyweb.dev/wiki/hajimu-iida.md)
- [Hao Li](https://pyweb.dev/wiki/hao-li.md)
- [Haoxiang Zhang](https://pyweb.dev/wiki/haoxiang-zhang.md)
- [Hiroshi Iwata](https://pyweb.dev/wiki/hiroshi-iwata.md)
- [Huy Nhat Phan](https://pyweb.dev/wiki/huy-nhat-phan.md)
- [Ivan Bercovich](https://pyweb.dev/wiki/ivan-bercovich.md)
- [Ivgeni Segal](https://pyweb.dev/wiki/ivgeni-segal.md)
- [Jean-Stanislas Denain](https://pyweb.dev/wiki/jean-stanislas-denain.md)
- [Joanna C. S. Santos](https://pyweb.dev/wiki/joanna-c-s-santos.md)
- [Joel Becker](https://pyweb.dev/wiki/joel-becker.md)
- [K M Ferdous](https://pyweb.dev/wiki/k-m-ferdous.md)
- [Ken’ichi Yamaguchi](https://pyweb.dev/wiki/kenichi-yamaguchi.md)
- [Kent Beck](https://pyweb.dev/wiki/kent-beck.md)
- [Kexun Zhang](https://pyweb.dev/wiki/kexun-zhang.md)
- [Kowshik Chowdhury](https://pyweb.dev/wiki/kowshik-chowdhury.md)
- [Leo Yu Zhang](https://pyweb.dev/wiki/leo-yu-zhang.md)
- [Maliheh Izadi](https://pyweb.dev/wiki/maliheh-izadi.md)
- [Martin Fowler](https://pyweb.dev/wiki/martin-fowler.md)
- [Mert Demirer](https://pyweb.dev/wiki/mert-demirer.md)
- [Miku Watanabe](https://pyweb.dev/wiki/miku-watanabe.md)
- [Minh Vu Thai Pham](https://pyweb.dev/wiki/minh-vu-thai-pham.md)
- [Model Evaluation & Threat Research](https://pyweb.dev/wiki/model-evaluation-and-threat-research.md)
- [Mohammed Latif Siddiq](https://pyweb.dev/wiki/mohammed-latif-siddiq.md)
- [Nate Rush](https://pyweb.dev/wiki/nate-rush.md)
- [Nghi D. Q. Bui](https://pyweb.dev/wiki/nghi-d-q-bui.md)
- [Peter Cihon](https://pyweb.dev/wiki/peter-cihon.md)
- [Prem Devanbu](https://pyweb.dev/wiki/prem-devanbu.md)
- [Rahul Pandita](https://pyweb.dev/wiki/rahul-pandita.md)
- [Razvan Mihai Popescu](https://pyweb.dev/wiki/razvan-mihai-popescu.md)
- [Sabrina Haque](https://pyweb.dev/wiki/sabrina-haque.md)
- [Sarvesh Ingale](https://pyweb.dev/wiki/sarvesh-ingale.md)
- [Shashwat Saxena](https://pyweb.dev/wiki/shashwat-saxena.md)
- [Shazibul Islam Shamim](https://pyweb.dev/wiki/shazibul-islam-shamim.md)
- [Shota Sawada](https://pyweb.dev/wiki/shota-sawada.md)
- [Sida Peng](https://pyweb.dev/wiki/sida-peng.md)
- [Simon Willison](https://pyweb.dev/wiki/simon-willison.md)
- [Stripe](https://pyweb.dev/wiki/stripe.md)
- [Tatsuya Shirai](https://pyweb.dev/wiki/tatsuya-shirai.md)
- [Tessl](https://pyweb.dev/wiki/tessl.md)
- [Tue Le](https://pyweb.dev/wiki/tue-le.md)
- [Vinicius Carvalho Lopes](https://pyweb.dev/wiki/vinicius-carvalho-lopes.md)
- [Wes McKinney](https://pyweb.dev/wiki/wes-mckinney.md)
- [Xinye Zhao](https://pyweb.dev/wiki/xinye-zhao.md)
- [Yanjun Zhang](https://pyweb.dev/wiki/yanjun-zhang.md)
- [Yi Liu](https://pyweb.dev/wiki/yi-liu.md)
- [Ying Zhang](https://pyweb.dev/wiki/ying-zhang.md)
- [Yubin Qu](https://pyweb.dev/wiki/yubin-qu.md)
- [Yuekang Li](https://pyweb.dev/wiki/yuekang-li.md)
- [Yutaro Kashiwa](https://pyweb.dev/wiki/yutaro-kashiwa.md)
- [Ziqian Zhong](https://pyweb.dev/wiki/ziqian-zhong.md)
- [Agentic Code Quality — Evidence Map](https://pyweb.dev/wiki/agentic-quality-evidence.md)
- [Agentic Code Quality Cycle 2 — Entity Accounting](https://pyweb.dev/wiki/agentic-code-quality-cycle-2-entity-accounting.md)
- [Agentic Code Quality Evidence — Entity Accounting](https://pyweb.dev/wiki/agentic-code-quality-entity-accounting.md)

### Evidence (verified primary sources)
- [addy-osmani-agentic-code-quality-2026](https://pyweb.dev/wiki/raw/articles/addy-osmani-agentic-code-quality-2026.md) | origin: https://addyosmani.com/blog/agentic-code-quality/ | ingested: 2026-08-27 | sha256: 7fa8c35a8f4c2cbb54d688cf503e4d9eb6d34e62247b93db5f8661df2881a7ee
- [tessl-coding-benchmarks-misaligned-system-harness-2026](https://pyweb.dev/wiki/raw/articles/tessl-coding-benchmarks-misaligned-system-harness-2026.md) | origin: https://arxiv.org/abs/2606.17799 | ingested: 2026-08-27 | sha256: 4b68e983226a2731dc5e89da3b27b4097495393d2208e4ad9d63c5aa68be3812
- [augment-code-software-factory-vs-devops-2026](https://pyweb.dev/wiki/raw/articles/augment-code-software-factory-vs-devops-2026.md) | origin: https://www.augmentcode.com/guides/software-factory-vs-devops | ingested: 2026-08-27 | sha256: 3c17a99bb8401309d43b2f5187ae78fa06093132e65287f3eb10d3f820253590
- [airbnb-eval-driven-development-2026](https://pyweb.dev/wiki/raw/articles/airbnb-eval-driven-development-2026.md) | origin: https://medium.com/airbnb-engineering/eval-driven-development-lessons-from-evaluating-genai-at-scale-e817e5ae5788 | ingested: 2026-08-27 | sha256: 94414a24787cabca4537e9a34113139762e66eb17d3996269377298569a51b0c
- [stripe-can-ai-agents-build-real-stripe-integrations-2026](https://pyweb.dev/wiki/raw/articles/stripe-can-ai-agents-build-real-stripe-integrations-2026.md) | origin: https://stripe.com/blog/can-ai-agents-build-real-stripe-integrations | ingested: 2026-08-27 | sha256: 3d30b4fb0d9bff438404d975b683b8ccba4ee0f626b36d10f70bf8eaf8bd025d
- [hamel-husain-shreya-shankar-evals-skills-2026](https://pyweb.dev/wiki/raw/articles/hamel-husain-shreya-shankar-evals-skills-2026.md) | origin: https://hamel.dev/blog/posts/evals-skills/ | ingested: 2026-08-27 | sha256: c4fc84c125acf75690e6f51dcf104f5c29eff8e6619d2bd33e8f9aee3642008b
- [simon-willison-agentic-engineering-patterns-2026](https://pyweb.dev/wiki/raw/articles/simon-willison-agentic-engineering-patterns-2026.md) | origin: https://simonwillison.net/guides/agentic-engineering-patterns/ | ingested: 2026-08-27 | sha256: f0fba6d5e10f26597d2b3fbe053cf17a297bbb041dae4b05f49a22ddea494e4c
- raw/papers/agent-generated-code-maintenance-2026.md (internal workspace doc)
- raw/papers/agentic-pull-requests-github-2026.md (internal workspace doc)
- raw/papers/swe-evo-long-horizon-2026.md (internal workspace doc)
- [metr-task-completion-time-horizons-2026](https://pyweb.dev/wiki/raw/articles/metr-task-completion-time-horizons-2026.md) | origin: https://metr.org/time-horizons/ | ingested: 2026-08-27 | sha256: 01fa095450fa59f3a2b8dcd70e5f5cd5a60cbd81ba7b9efd60f5c92b1e36790b
- [epoch-swe-bench-verified-analysis-2025](https://pyweb.dev/wiki/raw/articles/epoch-swe-bench-verified-analysis-2025.md) | origin: https://epoch.ai/publications/what-skills-does-swe-bench-verified-evaluate | ingested: 2026-08-27 | sha256: f05fee7402177a3d7e59f352333770bfbc66f47451ad08ad47c3822b8682bfaa
- [openai-swe-bench-verified-audit-2026](https://pyweb.dev/wiki/raw/articles/openai-swe-bench-verified-audit-2026.md) | origin: https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/ | ingested: 2026-08-27 | sha256: abbd740bbd2c6255f040fedde71b35764742bc503d3218c7921c7458b19824d3
- raw/papers/security-agentic-pull-requests-2026.md (internal workspace doc)
- raw/papers/code-review-agents-empirical-study-2026.md (internal workspace doc)
- raw/papers/tests-agentic-pull-requests-2026.md (internal workspace doc)
- raw/papers/adversarial-hacker-fixer-verifiers-2026.md (internal workspace doc)
- raw/papers/overeager-coding-agents-2026.md (internal workspace doc)
- [oxlint-type-aware-linting-2026](https://pyweb.dev/wiki/raw/articles/oxlint-type-aware-linting-2026.md) | origin: https://oxc.rs/docs/guide/usage/linter/type-aware.html | ingested: 2026-08-27 | sha256: f22fe71ea6055b9feebf5ebb1f934096b474cb07a57bb2e11ff1da3bee325dcf
- [dependency-cruiser-rules-reference-2026](https://pyweb.dev/wiki/raw/articles/dependency-cruiser-rules-reference-2026.md) | origin: https://github.com/sverweij/dependency-cruiser/blob/main/doc/rules-reference.md | ingested: 2026-08-27 | sha256: fc4d8ba1aa00e01bf5e657ff834a8732c8394f799bff30143b3b0900ae84d1e8
- [import-linter-layer-contracts-2026](https://pyweb.dev/wiki/raw/articles/import-linter-layer-contracts-2026.md) | origin: https://import-linter.readthedocs.io/en/v2.9/contract_types/layers/ | ingested: 2026-08-27 | sha256: d683a14684ad1223986b35ca14fc37707948676a989a55b8afb8db7710a15694
- [strykerjs-configuration-2026](https://pyweb.dev/wiki/raw/articles/strykerjs-configuration-2026.md) | origin: https://stryker-mutator.io/docs/stryker-js/configuration/ | ingested: 2026-08-27 | sha256: 0a38b0567a80f4f54f455c12e8ec0c631f10a5ebf97d6947e337abdf3fac56ab
- [mutmut-documentation-2026](https://pyweb.dev/wiki/raw/articles/mutmut-documentation-2026.md) | origin: https://mutmut.readthedocs.io/ | ingested: 2026-08-27 | sha256: 00e262ebaa5ca2bfc5875cb581f4fa2d5424be161dc03f61875a33fe413f4818

### Machine endpoints
- Knowledge graph: https://pyweb.dev/api/graph.json
- Graph analysis: https://pyweb.dev/api/graph-analysis.json
- Context index: https://pyweb.dev/llms.txt