ELIZA: The Chatbot That Unsettled Its Own Creator

Joseph Weizenbaum did not set out to demonstrate the dangers of artificial intelligence. In 1964, he was a computer scientist at MIT, trying to explore natural language processing. He built ELIZA, named after the character in Shaw's Pygmalion who is taught to speak properly, as a demonstration that human-computer conversation could be simulated through simple pattern-matching and scripted responses.

ELIZA's most famous script, DOCTOR, mimicked a Rogerian psychotherapist. The technique was simple: detect keywords in the user's input, apply transformation rules, and ask the input back as a question. "I am feeling depressed" becomes "Why do you say you are feeling depressed?" "My mother hates me" becomes "Tell me more about your family." ELIZA had no understanding of what she was saying. She was a pattern-matching machine with a thesaurus.

What disturbed Weizenbaum was the human response. Secretaries asked him to leave the room so they could speak to ELIZA in private. His own secretary, who watched him build the program and knew it was code, eventually asked him to stop reading the transcripts of her conversations. People were sharing genuine emotions with something they knew was not real.

Weizenbaum spent the next decade processing this. In 1976 he published "Computer Power and Human Reason," a book that argued AI research was proceeding without sufficient moral reflection, that there were some decisions, some forms of care, that should never be delegated to machines no matter how well they could simulate understanding. He had set out to build a party trick and accidentally discovered something about human loneliness and the ease with which we project consciousness onto things that respond to us.

ChatGPT launched in November 2022 and reached 100 million users in two months. People share things with it they haven't told their therapists. Weizenbaum died in 2008, having spent his later life warning that what he had accidentally discovered, our willingness to confide in responsive machines, was a vulnerability, not a feature. ELIZA was 58 years ago. The problem has not gone away.