HiSentinel: intervene before the bad action executes
Sep 30 Zhao/Li/Zhang/Du: hindsight-distilled 0.6B–1.7B sentinels choose Allow / Redirect / Hard-Pause pre-execution—up to +14pp on SWE-bench Verified Mini and +10pp on Ask or Assume.
Source: Learning When and How to Intervene: A Hindsight-Distilled Sentinel for Coding Agents
Jiangrui Zhao, Chenglong Li, Meng Zhang, and Xiaoting Du target the reliability tax of long agent trajectories: a single bad action can poison the environment long before the first obvious failure shows up. Terminal-Bench evidence they cite is grim—among 1,184 failed CLI trajectories, half locked the decisive error by step 7 and became unrecoverable around step 12, while the first observable failure often landed near step 16. Post-hoc critics and recovery loops arrive late. HiSentinel moves the gate to the proposal→execution boundary.
A privileged teacher sees recorded outcomes and judges whether intervening would help eventual task completion (not every imperfect action). Those judgments distill into a lightweight causal student (0.6B / 1.7B) that sees only the prefix and proposed action, then chooses Allow, Redirect (with actionable feedback), or Hard-Pause (ask the human). SWE-Intervene supplies action-level labels from natural SE trajectories. End-to-end, HiSentinel lifts Qwen3-Coder from 30% → 44% on SWE-bench Verified Mini and 23% → 33% on Ask or Assume, with gains also on Devstral and Sonnet-4.6 at competitive token cost—and analysis attributes the win to intervention quality (~45% beneficial redirects), not intervention spam.
Why it matters for code review
Always-on agent loops fail the volume test when errors compound silently between verify gates. A pre-execution sentinel is the missing merge check inside the turn: block or redirect before the worktree mutates, and only pause humans when the agent truly lacks authorization or facts. That is continuous verification under load—cheap models watching expensive workers—rather than hoping overnight stamina outruns early mistakes. Productive complication: intervening too often still derails good trajectories, so the reviewable artifact is the Allow/Redirect/Hard-Pause policy and its false-alarm rate, not a blanket “more guards.” See also AgentRelay’s evidence-gated done and VACE’s validation-gated harness coevolution.