Support Vector Machines: The Maximum Margin Classifier
Support Vector Machines dominated machine learning for a decadebuilt on a 1963 idea about the widest possible gap between classes.
Vladimir Vapnik developed the mathematical foundation of Support Vector Machines while working at the Institute of Control Sciences in Moscow in 1963. The core insight was geometric: given two classes of points in a high-dimensional space, the optimal decision boundary is the one that maximises the margin, the gap between the boundary and the nearest points (the support vectors) of each class. A wider margin means the classifier is more certain, and more likely to generalise to new data.
Vapnik emigrated to the United States in 1990 and joined AT&T Bell Labs, where he and Corinna Cortes published the modern SVM formulation in 1992. The practical breakthrough was the kernel trick: by mapping data into a higher-dimensional space through a kernel function, a linear classifier could separate data that was not linearly separable in its original space, without ever explicitly computing the high-dimensional representation. You could separate non-linear data with a linear boundary in a space you never directly visited.
Through the 1990s, SVMs became the dominant approach in machine learning. They had strong theoretical guarantees (VC dimension theory, structural risk minimisation), worked well on high-dimensional data, and outperformed neural networks on many benchmarks. At a time when neural networks were considered theoretically questionable, SVMs offered a principled alternative. They powered early spam filters, handwriting recognition, bioinformatics, and text classification.
AlexNet's 2012 ImageNet victory changed everything overnight. Deep neural networks, trained on GPUs with large datasets, outperformed SVMs on every vision task within a year. The paradigm shift was swift and total in computer vision. But SVMs never disappeared, they remain the method of choice in many domains where data is scarce, high-dimensional, or where theoretical guarantees about generalisation matter. Vapnik's margin intuition, developed in Moscow in 1963, still shows up in kernel methods at the heart of modern ML theory.
What it actually meant
SVMs dominated for a decade partly because they had theoretical guarantees at a time when neural networks did not. Theory enables trust, not because correctness requires proof, but because organisations need principled reasons to rely on systems. AlexNet's empirical dominance eventually displaced theory-driven choice, which is itself a lesson about how trust is built in a field.
What enterprises get wrong today
Enterprises over-rely on model cards, published benchmarks, and vendor evaluations as proxies for how a model will behave on their specific task. Benchmark performance on public datasets does not transfer to your data distribution. Third-party evaluations do not account for your domain, your label quality, or your edge cases.
Related capabilities
More from Algorithms
Ask about this story
SVMs were the dominant ML method for nearly a decade, theoretically sound, practically effective, and built on a 1963 idea about the widest possible gap between classes. Then deep learning made them look old-fashioned in two years. What would you like to explore?