Across 114,134 Java PRs, agent-supported changes add tests more often and delete them less, but agent test-only PRs cover less
Milanese et al. compare agent-supported and human-authored merged PRs within 391 Java repositories. Inside the same repo, agent PRs are 1.65× likelier to touch tests and rarely delete them; coverage of production changes looks the same, but agents' test-only PRs exercise less of the code they target.
Roberto Milanese, Francesco Salzano, Hovhannes Muradyan, Jingjing Wu, Remo Pareschi, Fausto Fasano and Mattia Fazzini look at who does the testing work when coding agents help write pull requests. They analyze 114,134 PRs merged between January 2025 and April 2026 in 391 Java repositories, detect five coding agents from PR- and commit-level signals, and compare agent-supported against human-authored PRs within each repository, using mixed-effects models that control for change size.
Pooled across all repos, both groups touch test code about equally often. Within a repository the picture flips: an agent-supported PR has 1.65× the odds of changing tests. Agent PRs add tests more often (odds ratios 1.50–1.58) and modify (0.70–0.76) and delete (0.51–0.77) them less. The tests they add are structurally similar and more often have an oracle. In the 95 repos the authors could build, coverage of production changes shows no difference — but in test-only PRs, agent tests cover less of the code they exercise. Test-smell differences are borderline and do not reproduce at the method level.
Why it matters for code review
A continuous agent pipeline that "always adds tests" looks healthy on a dashboard, and this paper says that much is real. What it does not buy is stronger verification: coverage of the change is flat, and the test-only PRs an always-on fleet is likeliest to open as busywork exercise less of their target. The tests agents rarely delete or rewrite also become maintenance the team inherits. For reviewers, the question on a green agent PR is not whether tests were added but whether they would fail without the change. The authors' other warning matters for anyone reading vendor metrics: pooled cross-project numbers can hide or reverse the within-repo effect. Read beside Peng et al. on agent performance PRs and Rahman et al. on evidence for AI-authored PRs.