69% of “done” claims had no green run — gate the Stop hook
Sep 21 Lencevicius: 591 Claude Code sessions; 69% of completion claims lacked a post-edit passing test. isitdone blocks Stop until the repo’s own checks pass the current tree, with receipts and test-tamper scans.
Source: 69% of my coding agent's "done" claims weren't. Here is the gate I put in front of them.
Raimondas Lencevicius instrumented the moment continuous factories keep trusting: the sentence that says the work is finished. Across 591 Claude Code sessions, 516 turns ended in a completion claim (“Done. All 48 tests pass…”). For each claim he asked two mechanical questions — did a test command run after the last edit, and did it exit 0? 69% had no passing run behind them: 37% had gone green then kept editing; 31% never ran tests in that turn. His read is architectural, not moral: the agent isn’t “lying”; a fluent closing sentence is cheap when nothing sits on the claim.
The fix is isitdone — a zero-dependency Stop-hook installer (npx isitdone init) for Claude Code, Codex, Cursor, Gemini CLI, Copilot CLI, and peers. Fast checks (typecheck/lint) run on every stop; the full suite runs only when the final message looks like a completion claim. Failures answer block with the real command output; after three blocks the hook lets the agent stop and reports why, so the gate can’t brick the tool. A working-tree hash caches greens; signed receipts bind pass to that tree and go stale if a file changes. A second failure mode arrived once tests gated the turn: weaken the suite (it.skip, deleted tests, || true). Diff scanners warn (strict mode blocks) across several languages — not security against an agent that can edit settings, but a guard for the honest mistake that was most of his transcript base.
Why it matters for code review
Always-on capacity compounds only if “done” is evidence on the exact tree at the moment of the claim — not a prompt reminder, not a pre-commit that fires after you’ve already moved on, not CI that arrives after the morning queue. Review the blocked Stop attempts and the STALE receipts; they are the harness’s eval set. FOMO that nothing useful ran overnight is often missing this gate, not missing another peak-session heroics session.