1901Algorithms
PCA: The Algorithm That Finds the Shape Hidden in Your Data
Karl Pearson invented PCA to draw the best line through a cloud of points. It became the most widely used dimensionality reduction technique in science.
Algorithms
Algorithms in AI History
The training algorithms, learning rules, and computational methods that turn data into intelligence.
15 stories
1967Algorithms
K-Means: The Forgotten Algorithm That Clusters Everything
K-means was published in an IBM report that almost no one read for 20 years.
1979Algorithms
The Bootstrap: Pulling Statistical Inference from the Data Itself
Bradley Efron realised you could estimate statistical uncertainty by sampling your own data, without knowing anything about the underlying distribution.
1986Algorithms
Backpropagation: The Algorithm That Would Not Be Published
Backprop was rejected by journals for years before it changed everything.
1991Algorithms
The Vanishing Gradient: Why Deep Networks Couldn't Learn, Until LSTM
The vanishing gradient problem sat unsolved for 10 years before LSTM fixed it.
1991Algorithms
Mixture of Experts: The Forgotten Architecture Behind the Frontier
Mixture of Experts was invented in 1991 and largely forgotten. Then it became the architecture reportedly used in GPT-4.
1992Algorithms
Support Vector Machines: The Maximum Margin Classifier
Support Vector Machines dominated machine learning for a decadebuilt on a 1963 idea about the widest possible gap between classes.
1998Algorithms
PageRank: When Link Structure Became a Markov Chain
PageRank treated the web as a Markov chain, and the math that ranked websites is now the backbone of Google's $2 trillion empire.
2001Algorithms
Random Forests: Wisdom of the Crowd, One Tree at a Time
Random Forests exist because one scientist was frustrated by noisy single trees.
2006Algorithms
Monte Carlo Tree Search: Random Sampling Meets Strategic Depth
Monte Carlo Tree Search made computers competitive at Go a decade before AlphaGo, by simulating thousands of random games.
2012Algorithms
Dropout: The Regularisation Trick Born in a Bank
Dropout was inspired by how banks prevent fraudnot letting tellers know everything.
2014Algorithms
Attention: The Mechanism That Rewired Everything
Attention was invented to help machines translate French novels.
2015Algorithms
Batch Normalisation: The Trick That Nobody Fully Understands
Batch Normalisation let researchers train networks 14× faster, and nobody fully understands exactly why it works.
2022Algorithms
RLHF: The Human Feedback Loop That Unlocked ChatGPT
RLHF, teaching language models to follow instructions by learning from human preferences, made ChatGPT possible.
2024Algorithms
Reasoning Models: When AI Learned to Think Before Answering
OpenAI's o1 didn't just answer questions: it paused to think. Spending compute at inference time rather than only at training time changed what "capable" meant for AI.