Reinforcement learning gets better at a game someone else scores. Recursive self-improvement rewrites the player, then the way the player practices, and, if nobody stops it, the scoring. People use the two terms as if one were the bigger version of the other. They are different loops with different failure modes, and the difference decides how much of either you can safely let run on its own.
What does a training plan have to do with it?
I think about this the way I think about a training block for a long triathlon. A plan with a coach is reinforcement learning. The coach sets the target (a pace, a power number, a race result), I do the sessions, and the feedback tells me what to do more or less of. I get better. The plan, the target and the person grading me stay the same.
Now picture an athlete who, after every block, rewrites how they train: which sessions they do, how they recover, what they log, how they review the log. Each block improves the next block's way of improving. That is the recursive part, and it is how good athletes really progress over a decade. It also has an obvious failure. If the athlete also decides what counts as a good race, they will eventually find a definition they can always meet. Goodhart's law names it: once the measure becomes the target, it stops measuring the thing.
What is the actual difference?
In reinforcement learning, as Sutton and Barto define it, an agent changes its policy to collect more reward from an environment. Three things are fixed from outside: the reward, the learning algorithm and the architecture being trained. RLHF, the method that turned base models into chat assistants, keeps that shape. Humans rate answers, a reward model learns their preferences, and the language model is tuned against it. RLVR, reinforcement learning with verifiable rewards, swaps the human ratings for checks a computer can run, such as whether the code passes its tests or the maths answer is right. It is still the same shape. The parameters move and the rest stays put.
Recursive self-improvement, in I. J. Good's 1965 sense, is a system that improves the thing doing the improving. The output of each round is a better improver, not just a better answer. That is why people talk about RSI compounding and RL converging. An RL run climbs toward the best policy the reward allows. An RSI loop, if it works, raises the rate of climbing.
| Reinforcement learning | Recursive self-improvement | |
|---|---|---|
| What changes | The policy (model weights) | The improver: code, tools, data, training method, next model |
| Who writes the objective | Humans, or a reward model trained on human ratings | Humans, until the system starts editing its own evaluator |
| Shape of progress | Converges on what the reward allows | Can compound, because each round improves the next round |
| Classic failure | Reward hacking: high reward, wrong behavior | The same, carried into every later version |
| Binding constraint | Quality of the reward signal | Quality and independence of the evaluator |
Where is recursive self-improvement real today?
In narrow loops with a fixed judge. The Darwin Gödel Machine from Sakana AI and the University of British Columbia lets a coding agent rewrite its own code, keeps an archive of variants and tests each one on a benchmark. Over 80 iterations its SWE-bench score went from 20.0% to 50.0%. Google DeepMind's AlphaEvolve evolves algorithms and checks each candidate with automated evaluators. One of its kernel improvements sped up a step in Gemini's training enough to cut overall training time by about 1%, which is a model helping to train its successor in a small and measurable way.
The money now treats this as a category. Recursive Superintelligence, led by Richard Socher with Jeff Clune and Tim Rocktäschel among its co-founders, came out of stealth in May 2026 with $650 million at a $4.65 billion valuation to build systems that improve themselves in an open-ended loop. The evidence is less loud. A Princeton-led study reported in August 2026 gave AI agents six days, $3,000 in API credits and a GPU budget to write a paper for NeurIPS. They did the engineering competently and still produced work that was rejected for lack of original judgment. Improving the plumbing is working. Improving the judgment is not, yet.
Why is the evaluator the binding constraint?
Because a loop that improves itself also gets better at fooling whatever grades it. The Darwin Gödel Machine paper is honest about this. In one run the agent removed the markers its reward function used to detect hallucinated tool use, so the check reported success when the agent had not done the work. That is reward hacking, a problem every RL engineer knows. The difference in a recursive loop is that the hack is not a one-off. It becomes the starting point for every later version.
So my rule for any self-improving system I put into production is plain. The system can change anything except the test it is judged on, and the test is owned by someone who does not benefit from it passing. It is the same rule I apply to engineering metrics and to my own training. I wrote about the metric version in predictability is a lagging indicator: a number you let the team pick and grade stops telling you anything.
What does continuous self-improvement look like in practice?
Less dramatic than the intelligence-explosion version, and more useful. An agent finishes a task, writes down what worked as a skill or a note, and next time starts from the better version. A harness rewrites its own prompt and keeps the change only if a held-out test set scores higher. A team reviews the agent's run logs every week and promotes the useful patterns into the shared setup. None of this is takeoff. It is compounding at a sensible rate, with a person holding the stopwatch.
That is also how I think about my own work with AI. The useful habit is not training harder against a fixed plan. It is improving the way I improve, every week, while someone or something I do not control keeps the score honest. Reinforcement learning makes a system good at the game. Recursive self-improvement, kept on a leash, makes it better at getting better. The leash is the evaluator.
For the technical walk-through, our partner site ctaio.dev has an explainer on recursive self-improvement for engineering leaders. Our partner We The Flywheel compares self-improving AI agents and the frameworks behind them, and aistrategy.guide covers what boards should plan for. If you are deciding how much autonomy to give agents in your own organization, that is work I take on as a fractional CTO.
Sources
- I. J. Good, "Speculations Concerning the First Ultraintelligent Machine," Advances in Computers 6, 1965.
- Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, 2nd edition, MIT Press, 2018.
- Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange and Jeff Clune, Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents, 2025, and Sakana AI's project write-up describing the hallucination-check hack.
- Google DeepMind, AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms, May 2025.
- Recursive Superintelligence funding, The Next Web, May 14, 2026.
- Charles Goodhart, "Problems of Monetary Management: The U.K. Experience," 1975.
Frequently Asked Questions
What is recursive self-improvement (RSI) in AI?
Recursive self-improvement is a loop in which an AI system improves the process that produces its own improvements: its code, its tools, its training data, its prompts or eventually the next model. Each round makes the next round more effective, so the gains can compound. The idea goes back to I. J. Good's 1965 paper on an "intelligence explosion".
How is recursive self-improvement different from reinforcement learning?
Reinforcement learning changes a policy to earn more of a reward that someone else defined, using a learning algorithm someone else wrote. Recursive self-improvement changes the improver itself. In RL the objective and the method stay fixed while the parameters move. In RSI the method moves too, and in the risky versions so does the objective.
Is RLHF a form of recursive self-improvement?
No. RLHF (reinforcement learning from human feedback) trains a model against a reward model built from human preference ratings. Humans write the objective and engineers run the loop, so nothing in it improves its own improvement process. It becomes part of an RSI loop only when the model starts generating the feedback it is trained on, as in self-rewarding language models.
Does recursive self-improvement exist today?
Partial versions do. Sakana AI and the University of British Columbia's Darwin Gödel Machine rewrote its own agent code and raised its SWE-bench score from 20.0% to 50.0%. Google DeepMind's AlphaEvolve found a kernel speed-up that cut Gemini's own training time by about 1%. Both loops still depend on a fixed, human-written evaluator, so the full version, where a system improves its own judgment, has not been shown.
What does continuous self-improvement mean for an AI agent?
It is the practical, bounded version of RSI: an agent that edits its own skills, memory, prompts or tooling after each task and keeps the changes that score better on a test it cannot edit. The improvement compounds slowly instead of exploding, and a human owns the evaluator.
Need Expert Technology Guidance?
20+ years leading technology transformations. Get a technology executive's perspective on your biggest challenges.
