Adoption of AI coding tools is no longer a debate — it's close to universal. 92% of US developers report using AI as a "daily companion" in 2026 (WebSearch synthesis, Index.dev), and GitHub Copilot alone has passed 20 million all-time users and 4.7 million paid subscribers as of January 2026 (Index.dev synthesis). The Stack Overflow 2025 Developer Survey found 84% of respondents using or planning to use AI tools, with 51% of professional developers using them daily (Index.dev synthesis).
What's not settled — and this is the actual story in 2026 — is whether all that usage translates into measurably faster, better software. The vendor-reported numbers and the one rigorous controlled study on the topic disagree sharply.
The productivity numbers vendors report
Self-reported and vendor-adjacent studies paint a strongly positive picture:
- 81% of GitHub Copilot users report a productivity boost for coding and testing tasks (Index.dev synthesis)
- Developers using Copilot implemented a benchmark server task 55.8% faster than a control group (Index.dev synthesis)
- Teams using "Level 3 agentic" AI tools (autonomous multi-step coding agents, not just autocomplete) report 10–20x velocity gains and 40–78% fewer bugs (Index.dev synthesis)
- JetBrains' 2025 survey found ~85% regular AI usage among respondents, with 62% relying on at least one AI assistant or agent as part of their normal workflow (Index.dev synthesis)
But even within these self-reported numbers there's an internal contradiction worth flagging directly:
Warning
METR's controlled study: the most rigorous data point available
Most "does AI make developers faster" claims come from self-report surveys or vendor-commissioned benchmarks — both prone to selection and motivation bias. METR (a research organization focused on AI capability evaluation) ran an actual randomized controlled trial: experienced open-source developers were randomly assigned to complete real tasks either with or without AI tools, paid $150/hour, between February and June 2025.
The result was the opposite of the vendor narrative: AI tool use caused a 20% slowdown in task completion time among experienced developers working on codebases they knew well (METR).
METR ran a follow-up starting August 2025 to check whether this held up, and the picture got messier rather than clearer:
- The original developer cohort showed an estimated speedup of -18% (confidence interval: -38% to +9%) — still net negative, but wide enough that "no effect" is within range (METR)
- Newly recruited developers showed -4% (confidence interval -15% to +9%) — closer to neutral (METR)
- METR itself flags the true effect "could be much higher" (i.e., AI could be genuinely helpful) due to a specific selection-bias problem it identified in its own study design (METR)
Why METR's own numbers might understate AI's benefit
This is the most methodologically honest part of the 2026 landscape: METR published the flaws in its own follow-up rather than just the headline number. The problems it identified:
- Participation selection bias — developers increasingly declined to take part in the no-AI condition because they "do not wish to work without AI," which systematically excludes from the study exactly the developers who'd show the biggest AI benefit (METR)
- Task selection bias — participants avoided submitting tasks they expected AI to complete quickly, skewing the task mix toward AI's weaker cases (METR)
- Measurement noise — developers running concurrent AI agents made clean time-tracking unreliable (METR)
- Quality variance — output quality differed between the AI-allowed and AI-disallowed conditions, meaning "time to completion" alone doesn't fully capture what changed (METR)
METR is now moving away from this randomized-task design toward observational data analysis, surveys, fixed-task designs, and direct agent capability evaluations (METR) — an implicit admission that a clean RCT on "does AI make you faster" is harder to run well than it looks.
The code-quality side of the ledger
Separately from speed, there's now real data on what AI-generated code looks like once it ships. DX's Q4 2025 impact report, covering over 135,000 developers, found 91% AI adoption within its sample and that 22% of all merged code is now AI-authored (WebSearch synthesis, GetPanto).
But that code carries a measurable quality cost: CodeRabbit's December 2025 analysis found AI-coauthored PRs show roughly 1.7x more review issues than human-only PRs (WebSearch synthesis). A separate MSR 2026 paper on post-merge code quality found that even AI agent PRs that get merged introduce code smells and maintainability issues at rates human-authored PRs in the same repos do not (WebSearch synthesis).
Note
Reconciling the contradictory numbers
| Claim | Source type | Direction |
|---|---|---|
| 81% report productivity boost | Self-report survey | Positive |
| 55.8% faster on benchmark task | Vendor-adjacent benchmark | Positive |
| 78% feel more productive, tasks take 19% longer | Mixed self-report + measured | Contradictory |
| 20% slowdown (original METR RCT) | Controlled experiment | Negative |
| -18% to -4% (METR follow-up, contested) | Controlled experiment, bias-flagged | Negative but uncertain |
| 1.7x more PR review issues | Code-review analysis | Negative (quality) |
| 10-20x velocity for agentic tools | Vendor case studies | Strongly positive |
The pattern: self-reported "feels faster" and vendor case studies skew strongly positive; the one controlled experiment (METR) skews negative but with acknowledged methodological problems; and independent code-quality analysis suggests the code that does ship faster is measurably lower quality on average.
The likeliest honest read: AI tools speed up some tasks substantially (boilerplate, well-specified agentic tasks, greenfield scaffolding) while slowing down or degrading quality on others (unfamiliar codebases, tasks requiring deep context the model doesn't have, situations where review rigor drops because the code "looks right"). Aggregate numbers in either direction average over that split and mislead by omission.
Practical takeaway
Don't trust a single number in either direction. If you're evaluating AI coding tools for your team: measure your own task mix rather than importing an industry-wide average, and don't relax code review standards just because AI wrote the PR — the 1.7x-more-issues finding suggests that's exactly backwards. The METR study's most useful contribution isn't its headline number, it's the reminder that "developers feel faster" and "developers are faster" are different claims, and 2026's data still can't fully reconcile them.
Sources: METR — Changing Our Developer Productivity Experiment Design, Index.dev — Top 100 AI Pair Programming Statistics 2026, GetPanto — AI Coding Statistics, Tenki — AI Code Creates 1.7x More Review Issues
Get new posts as they publish
No spam — just the next post, straight to your inbox.