You don’t need AGI for autonomous improvement — you need three things: a quantifiable metric, atomic changes, and automatic rollback. In Karpathy’s Autoresearch, every experiment commits before verification, then either advances the branch (if val_bpb improved) or resets to the previous state. This makes failure cheap and reversible, which is what makes aggressive exploration safe.
This sharpens the insight from Autonomous coding loops need small stories and fast feedback to work: the Ralph pattern identified “small stories and fast feedback” as the enablers of autonomous loops, but the deeper enabler is the rollback safety net. Fast feedback tells you whether a change worked; automatic rollback ensures that failures cost nothing. The combination means the system can run indefinitely without human supervision — not because the agent is intelligent enough to avoid mistakes, but because mistakes are instantly undone. This is why Verification is the single highest-leverage practice for agent-assisted coding has such outsized impact — and why A mediocre agent inside a strong harness outperforms a stronger agent inside a messy one holds: the safety machinery matters more than the reasoning capability.