Least Squares: The Statistical War That Shaped Every Regression
Gauss claimed he invented it. Legendre published it first. The dispute never fully resolved, and it's still the most widely used statistical technique in history.
On January 1, 1801, the astronomer Giuseppe Piazzi discovered a small celestial body he named Ceres and tracked it for 41 days before it disappeared behind the sun. The question of where it would reappear seemed unanswerable with the mathematics of the day. Carl Friedrich Gauss, a 24-year-old in Brunswick, claimed to have a method for fitting the best possible curve to imperfect measurements. He predicted Ceres' location. When telescopes found it again, within a degree of his prediction, Gauss became famous overnight.
The method Gauss used was the method of least squares: find the values that minimise the sum of squared differences between predicted and observed measurements. It handles noise in data by finding the curve that comes as close as possible to all points simultaneously, rather than interpolating exactly through any subset. In 1809, Gauss published it in Theoria Motus Corporum Coelestium, but Adrien-Marie Legendre had already published an appendix describing the identical method in a book on comets in 1805. A priority dispute began that neither man lived to resolve.
Legendre was furious. He had published first. Gauss claimed private use since 1795, which Legendre found convenient and unprovable. The mathematical community never reached a unanimous verdict. Gauss's formulation included a statistical justification, connecting least squares to maximum likelihood estimation under Gaussian noise, that Legendre's had lacked. In mathematics, the first publication does not always win if the second is more rigorous.
Francis Galton introduced the word "regression" in the 1880s while studying the heights of fathers and sons. He noticed that sons of unusually tall fathers were, on average, shorter than their fathers: regressing toward the mean. The statistical method he used was least squares. The word "regression" got attached to the technique through this particular application. Every time a data scientist runs a "linear regression," they are using Legendre and Gauss's method, named after Galton's plant observations, to solve problems neither man could have imagined.
Linear regression remains the most widely taught and widely deployed statistical method in history. Generalised Linear Models extend it to binary outcomes (logistic regression) and counts (Poisson regression). Ridge regression and LASSO add regularisation to handle high-dimensional data. And ordinary least squares is the direct ancestor of the loss functions used to train neural networks: mean squared error is least squares applied to neural network weights. The method Gauss used to find a lost asteroid in 1801 is still the default starting point for predictive modelling in 2025.
What it actually meant
Least squares is the solution to a specific problem: minimise squared error. Whether minimising squared error is the right objective depends on whether your errors are actually Gaussian, whether you care more about large errors than small ones, and whether outliers are informative or corrupting. The method that found a lost asteroid works because astronomical measurement errors happen to be approximately Gaussian. Your enterprise data may not share that property.
What enterprises get wrong today
Teams default to mean squared error and linear models on all regression problems without checking whether the loss function matches the business objective. If you care about median prediction error, or if your distribution has heavy tails, or if large errors cost disproportionately more than small ones, minimising squared error optimises the wrong thing. The default is not neutral: it encodes specific assumptions about your error distribution.
Related capabilities
More from Mathematics
Ask about this story
Gauss found a lost asteroid using least squares, then claimed he'd invented the method years before Legendre published it. The dispute never resolved. That same technique is now the foundation of every linear model and neural network loss function in use today. What would you like to explore?