Faros: AI review found critical issues in all 28 human-approved PRs, then a fix loop ran all night
Oct 28: replaying merged PRs through two independent model reviewers found 3 to 14 critical issues each behind one-word LGTMs. The same harness also showed how an unattended review loop fails to converge without a severity cutoff.
Source: What AI caught in PRs our human reviewers had approved
Faros head of product Thierry Donneau-Golencer replayed 28 merged pull requests that humans had already approved through a new review harness built on Open Code Review. Two reviewers run in parallel, one on an Anthropic model and one on an OpenAI model, with no shared context on the first pass, and findings are rated P0 to P4. Every one of the 28 came back with between 3 and 14 critical issues the human review hadn't raised. Most of the original approvals were a single word: LGTM. One live catch was cached input being double-counted, reporting $5.15 in savings instead of $1.15.
Why it matters for code review
The second half is the part worth keeping for anyone running review continuously. An AI reviewer always finds something. A helper skill auto-applied every "fix now" finding, including low-severity ones, so each fix changed the diff, each new diff earned a fresh review, and each fresh review found more small things. One run went through the night and into the next morning until a VPN drop stopped it. A documentation PR went 13 rounds on P2 and P3 findings alone. The gate said P0 and P1; the loop didn't.
Their fix was to make the loop and the gate the same rule: once every reviewer clears P0 and P1, the PR is approved, and lower findings stay on the PR as a record with a person's disposition. They also made runs stoppable, made infrastructure errors end a run instead of retrying, and log the cost of every round. Steady overnight review only compounds if it knows when it's done. Still open for them: whether a human should ever be allowed to merge over an unresolved P1 without a written dismissal.