Autonomous agents lift commits 240% but releases only 30%, because human steps are the weak link
Demirer, Musolff and Yang match AI usage telemetry to more than 500,000 GitHub developers. Each generation of tool adds more coding activity, but the gains shrink at every step from commits to projects to releases, consistent with human review and judgment being hard to substitute.
Source: Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools
Mert Demirer, Leon Musolff and Liyuan Yang ask how AI coding gains change across tool generations, and how much of the gain reaches shipped software. Their NBER working paper (May 2026, revised September 2026) combines data on more than 500,000 GitHub developers with AI usage telemetry, and uses a matched event study to compare adopters against similar non-adopters.
Commits rise with each generation. The cumulative effect is 30% for autocomplete, 180% once developers add interactive agents that work alongside them (the paper's example is Claude Code), and 240% with autonomous agents that run until a task is done (its example is GitHub's agent). But the gains shrink further up the chain. The 240% effect on commits falls to 80% on the number of projects and 30% on actual releases. The authors read this as a "weak link" pattern, with an estimated elasticity of substitution of 0.23 between AI and human effort, meaning the two are strong complements. They note that async agents produce entire pull requests, which still need human review before merge. Across four software marketplaces they find a sharp rise in new apps but no rise in total usage.
Why it matters for code review
This is the large-sample version of what many factory write-ups report one team at a time. An agent that runs all night does raise output, but output measured as commits or PRs is not what ships. The steps that turn a PR into a release, mostly review and the decision to ship, didn't scale with it. A harness that only adds more overnight work pushes harder on the step that is already holding things up. The useful question for anyone running agents continuously is how much of the human gate that harness actually removes or speeds up, since that is where the 240% turns into 30%.