Bajwa: a month of traced agent pipeline runs, where twelve loops in three hours looked like progress
Rayed Bajwa traced every agent turn in his spec-to-merge pipeline for a month. One feature looped implement, review and verify twelve times while the report never moved, 73 of 303 handoffs failed basic checks, and 82% of verify passes changed nothing.
Source: What Broke When I Let AI Agents Run My Software Process for a Month
What Broke When I Let AI Agents Run My Software Process for a Month is Rayed Bajwa's write-up of running Spaces, his own source-available pipeline, on his projects for a month. Each stage is a separate agent (research, spec, plan, tasks, implement, review, verify, deliver), a person approves between stages, and every agent turn is traced. In one week, 96 runs produced 303 stage-to-stage handoffs he could inspect.
A loop that looked like work. One afternoon a mobile feature went through implement, review and verify twelve times in three hours. Review approved each time. The verification report said the same thing every pass: 5 of 15 acceptance criteria met. The other ten were checks only a person holding a phone could do. The pipeline's rule was "while any implementation task is open, implement again," and it had no idea a task could belong to a human. His fixes: end the loop when a verify pass meets the same number of criteria as the last one, mark person-only tasks explicitly, and recommend "accept and finish" once nothing is left for an agent.
Summaries drop the identifiers. He wrote six checks for the handoff summaries and ran them over the week. 73 of 303 failed at least one. Of the references that went missing, 68 were commit hashes, 10 were PR numbers and 6 were spec files. None kept the verification status line. The fix is to let a model summarize prose but pull hashes, PR numbers, paths and statuses out of the raw output in code.
Long runs outlive their credentials. A run that lasted longer than an hour carried an expired GitHub App token, and the failed push wrote the auth header into the run log because the token was passed on the command line. Credentials now go through the environment, are refreshed every ten minutes, and errors log scrubbed stderr.
Score runs as they happen. He found the twelve-loop afternoon by reading logs, so every run is now scored for handoff quality, report format and loop progress. Before the fixes, 82% of verify passes recorded no change in criteria met.
Why it matters for code review
A pipeline that keeps running is only useful if each pass moves something a reviewer can check. Bajwa's numbers show how an always-on loop can stay busy and green while making no progress, and that most of what went wrong was catchable with simple deterministic checks rather than another model's judgment. He is the builder of the tool he's reporting on, and the figures come from his own projects.