Chollet: Science as recursively self-improving system shows AI RSI won't cause intelligence explosion

The real world abounds with recursively self-improving systems, but one in particular deserves attention: science, modeled as a system (perhaps even as an agent, with goals and resources). If you want to truly understand AI recursive self-improvement (RSI), science should be your reference point.

Science as a Recursively Self-Improving System

Science is an intelligent system, and it is obviously recursively self-improving:

1. Scientific discoveries unlock new technology that helps build better experimental tools. This is a top driver of progress in nearly all fields.
2. They unlock new conceptual advances (ideas, theories) that help solve more problems.
3. They increase society's economic output, leading to more resources flowing into science.
4. They unlock better, faster tooling (e.g., more compute via better chip and networking technology).

Exponential Growth in Scientific Inputs

As a result, many measures of scientific input grow exponentially:

• Headcount – doubles every ~15 years
• Global R&D spending – doubles a bit faster, every ~13 years
• Papers and patents – technically a measure of headcount
• Compute dedicated to science – doubles every ~2 years

But Is Scientific Progress Exponential?

Historically, the rate of scientific impact over time has remained roughly constant since the start of the Industrial Revolution — meaning scientific progress is linear.

Consider the following periods:

1850–1900: Evolution, germ theory & antiseptic surgery, thermodynamics, electromagnetic field equations, the periodic table, pharmaceuticals, electricity, telegraph and telephone, internal combustion engine, skyscrapers, mechanized agriculture...

1900–1950: Special and general relativity, quantum mechanics, nuclear fission & atomic energy, antibiotics, genetic theory, electronic computers, information theory, synthetic polymers and plastics, the transistor, aviation...

1950–2000: DNA, genetic engineering, integrated circuits, microprocessors & personal computing, the Internet, crewed spaceflight, moon landing, satellite communications & GPS, standard model of particle physics...

In real terms — like life expectancy, which has increased in a remarkably linear fashion of roughly 3 months per year since 1840 — progress is a straight line. This is especially apparent for fields where impact is easy to measure, such as biology, medicine, and agriculture.

Confirming Evidence

I first wrote about this phenomenon and its causes in 2012, and a steady stream of research has confirmed it in the years since. Examples include:

• The 2018 paper by Nielsen and Collison, "Science Is Getting Less Bang for Its Buck"
• The 2020 economic paper, "Are Ideas Getting Harder to Find?"

(In fact, I believe the Nielsen paper stemmed from a conversation I had with him about this exact idea six months earlier.)

The Root Cause

In short, the primary cause is that research solves the highest-impact, easiest problems first, and every subsequent problem is either harder or lower-impact — exponentially so. The paper that presented information theory wasn't very hard to write (single author!), but you would have a hard time ever writing a computer science paper that beats it in impact.

This is why science as a system requires exponential resources (input) to produce linear impact (output). It gets exponentially harder over time.

Implications for AI RSI

Worth thinking about if you are pondering RSI for AI. I fully believe AI RSI is already happening and will accelerate in the future. But I do not believe this leads to an "intelligence explosion" — that would fly in the face of everything I know about intelligence and everything I know about recursively self-improving systems.