---
title: "We are Changing our Developer Productivity Experiment Design - METR"
section: "raw"
type: "source"
created: "2026-08-27"
updated: "2026-08-27"
canonical: "https://pyweb.dev/wiki/raw/articles/metr-uplift-update-2026"
---
# We are Changing our Developer Productivity Experiment Design - METR

**Authors:** Joel Becker, Nate Rush, Tom Cunningham, David Rein, Khalid Mahamud
**Date:** 2026-02-24
**URL:** https://metr.org/blog/2026-02-24-uplift-update/

## Abstract / Summary
METR follow-up study on developer productivity with late-2025 AI tools. Notes severe selection bias: developers refusing to work in AI-disallowed condition, 30-50% selective task submission, pay rate dropped from $150/hr to $50/hr. Original developer subset showed raw -18% completion time (speedup) but confidence interval spanned -38% to +9%. Newly recruited devs showed -4% (-15% to +9%). Concludes data is only very weak evidence for true effect size.

---

## Agent Navigation

### Machine endpoints
- Knowledge graph: https://pyweb.dev/api/graph.json
- Graph analysis: https://pyweb.dev/api/graph-analysis.json
- Context index: https://pyweb.dev/llms.txt