The Bitter Lesson: Why Scale Keeps Beating Cleverness
Richard Sutton argued that 70 years of AI showed one lesson: compute always wins over human knowledge.
In March 2019, Richard Sutton, one of the founders of reinforcement learning, published a short essay titled "The Bitter Lesson." In 1,200 words, he argued that the history of AI research, surveyed honestly, told a consistent and uncomfortable story: methods that leveraged general-purpose computation and learning almost always beat methods that encoded human domain knowledge. And they beat them more completely as compute improved over time.
Sutton traced the pattern across chess (evaluation functions vs. search), speech recognition (linguistic models vs. statistical models trained on data), computer vision (feature engineering vs. convolutional networks), and game-playing. In each domain, the "smart" approach, encoding human knowledge about the domain into the system, held its lead while compute was expensive. Then, as hardware got cheaper and datasets grew, the "scaling" approach caught up and surpassed it. The bitter lesson: researchers who spent careers encoding domain knowledge watched their work become obsolete when compute arrived in sufficient quantity.
The essay was polarising. Yann LeCun and others pushed back: inductive biases (like the convolution operation in CNNs) are themselves a form of knowledge encoding that scales, they are not opposed to learning, they make learning efficient. Sutton's dichotomy between "human knowledge" and "learning from computation" was too sharp. And yet the essay captured something real: the systems that dominated 2023, large language models trained on internet-scale data with minimal architecture priors, were exactly the kind of "pure scaling" approaches Sutton described.
"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin." Sutton wrote this in 2019, one year after GPT-1 and two years before GPT-3 made the point undeniable. Whether the lesson is bitter depends on what you were hoping to be true. For researchers who believed intelligence required careful encoding of human knowledge, it was bitter. For those who bet on scale, it was vindicating.
What it actually meant
Sutton was describing a historical pattern, not prescribing an approach. The lesson is not "do not use domain knowledge." It is "do not assume domain knowledge compensates for insufficient data and compute." The right response is to invest expertise in data curation, evaluation design, and production monitoring, where it cannot be replaced by scale.
What enterprises get wrong today
AI teams that read the Bitter Lesson conclude domain experts are not needed and that more data is the answer. But the domain expert's role shifts rather than disappears: from feature engineering to quality control, from model building to evaluation design, from architecture to production monitoring. Removing domain expertise removes the ability to catch the model when it is wrong in the ways that matter.
Related capabilities
More from People
Ask about this story
Sutton's Bitter Lesson argued that 70 years of AI showed one pattern: compute + learning always beats human knowledge encoding, eventually. Written in 2019, it reads differently now that GPT-4 exists. What would you like to explore?