The Perceptron: A Machine That Would Think

On July 8, 1958, the New York Times ran a story under the headline "New Navy Device Learns By Doing." The device was the Perceptron, a machine built by Frank Rosenblatt at Cornell Aeronautical Laboratory, funded by the Office of Naval Research, and housed in a room-sized cabinet of wires, motors, and photocells. The Times reported that the Navy expected it would "be able to walk, talk, see, write, reproduce itself and be conscious of its existence."

Rosenblatt's Perceptron was inspired by a 1943 paper by Warren McCulloch and Walter Pitts, who had modelled the neuron as a simple binary threshold function. If the weighted sum of inputs exceeded a threshold, the neuron fires; if not, it doesn't. Rosenblatt showed that a machine built from many such neurons, connected and trained on examples, could learn to classify patterns, including distinguishing simple shapes and letters from a camera input.

The training algorithm was equally novel. Given a labelled example, if the Perceptron got it wrong, adjust the weights slightly toward the correct answer. Get it right, do nothing. Repeat across thousands of examples. Rosenblatt proved mathematically, via the Perceptron Convergence Theorem, that if a linearly separable classification existed, his algorithm would find it in finite time.

The problem was that not everything is linearly separable. In 1969, Marvin Minsky and Seymour Papert published "Perceptrons"a rigorous mathematical critique that demonstrated the Perceptron could not solve XOR or any non-linearly-separable problem. The book triggered a decade-long collapse in neural network funding. Rosenblatt himself died in a boating accident in 1971, before seeing the revival of the ideas he had championed.

The Perceptron's legacy is paradoxical. It was simultaneously overhyped beyond what it could deliver and underestimated as the seed of what would follow. Today's deep learning networks are stacks of perceptron-like units trained with the algorithm Rumelhart and Hinton refined two decades later. The room-sized Navy device that was going to be conscious turns out to have been the first layer of something that actually, eventually, came close.