In 7,581 Copilot agent PRs, 58% saw no human at all, and the first intervention is the one that counts
Sohail, Kurmantayev, Mahmoud and Nadi study real Copilot coding-agent sessions instead of benchmark runs. Most PRs run untouched; when developers step in, it's brief, and only the first nudge measurably shortens what the agent does next.
Izah Sohail, Yerdanat Kurmantayev, May Mahmoud and Sarah Nadi point out that most coding-agent studies assume one autonomous run against a benchmark, which is not how people actually use these agents. So they study 7,581 pull requests authored by GitHub's Copilot coding agent whose session data is public, and trace how humans and the agent trade turns inside each one.
58% of the PRs had no human intervention at all. When developers did step in, the collaboration was short: a median of two human-agent loops. Merged PRs tended to start with fewer agent iterations than unmerged ones. The trajectory right after the first human intervention dropped by 20.8 iterations and 8.4 minutes; later interventions barely moved iteration counts and didn't change duration significantly. The authors' reading is that most of the work happens in the initial run, and that many interventions are small, focused requests the agent finishes quickly. They also show developers sometimes have to steer the same PR repeatedly, or across several versions of it.
Why it matters for code review
This is useful data for anyone running agents unattended. Most of these PRs already ran without a person, and a long first run was a warning sign rather than effort to admire: merged PRs started shorter. The cheap human move is one early, specific correction, not a long session of back-and-forth. For an always-on setup, that argues for a review step that lands soon after the first push, while a single comment still changes the outcome, rather than a morning pile where every PR has already run to the end on its own. It also complicates the idea that a harness can replace the human entirely: the one intervention that matters is still a human one.