Paper

In agent harnesses, a separate critic beats making the agent check its own work

A Cornell and Alibaba study compares 17 ways to add verification to a coding agent's harness across seven models. A fresh-context critic that reviews the changes adds 4 to 10 points of accuracy at little extra cost, while gates that reopen the agent's own work cost up to 1.76x for smaller gains.

Source: Rethinking Verification in Agent Harnesses

Zhengxin Zhang, Erkang Zhu, Yin Lin, Claire Cardie, Bolin Ding, and Jingren Zhou treat verification as a design choice inside the harness, the loop that runs a coding agent, rather than as something the model either does or doesn't do. They sort the options into four places checking can enter: instructions before the task starts, reminders during work, completion gates that reopen the work when the agent tries to stop, and independent review by a separate critic in a fresh context that can run its own checks but can't edit the deliverable.

They compare 17 designs across seven frontier coding models on Terminal-Bench and SWE-bench Verified, plus an internal set of 18 visual and interactive tasks judged blind by people.

The baseline is worth knowing first: left alone, agents ran a recognizable check after only 6.2% to 13.6% of their editing turns.

Instructions and reminders helped a little and unevenly by model. Completion gates helped more but were expensive: designs that reject the agent's first "done" cost 1.60x to 1.76x baseline on Terminal-Bench. The independent critic was the best trade. It raised average accuracy by 5.9 to 9.5 points on Terminal-Bench and 3.9 to 6.6 on SWE-bench Verified, at 1.13x to 1.25x and 1.00x to 1.04x baseline cost, counting the critic and the agent's revisions. Giving the critic just the task and the diff did as well as giving it the agent's full trajectory, which raised critic cost by 29% to 43%. Adding rounds or critics bought up to 3.5 more points, and four parallel critics nearly matched four sequential rounds at about 19% lower total cost.

Why it matters for code review

This is the code review argument run inside the agent loop. The checks that help most come from someone who didn't write the change, looking at the change against the task, before it's accepted. Telling the author to be more careful, or sending it back to re-check its own work, buys less per dollar.

For teams running agents continuously, the cost numbers matter as much as the accuracy. A cheap critic reading diffs is overhead you can afford on every change, all night. Rejection loops that send the agent back to work are the kind of verification tax that grows with volume. One caution from the paper itself: a second review round lifted one model by about 11 points and lowered another by about 2, so the right amount of review depends on the model you run.