Stable Diffusion: How Open Weights Changed Generative AI

In April 2022, OpenAI released DALL-E 2, a text-to-image system that produced startlingly realistic images from natural language descriptions. Access was by waitlist. Google released Imagen the same month, but only for researchers. Midjourney launched as an invite-only Discord bot. The quality was astonishing; the access was tightly controlled. Then, in August 2022, Stability AI released Stable Diffusion with open weights. You could download the model and run it on your own GPU. Everything changed.

The architecture was latent diffusion, developed by Robin Rombach, Andreas Blattmann, and colleagues at Ludwig Maximilian University of Munich. Instead of applying diffusion directly in pixel space (adding and removing noise at full image resolution), latent diffusion compressed the image into a lower-dimensional latent representation, applied diffusion there, then decoded back to pixels. This made generation dramatically cheaper and faster without significant quality loss. The key insight was doing the hard work in a small space, not a large one.

The open weights enabled an ecosystem that no closed model could match. Within weeks, researchers had built ControlNet (adding spatial control to generation), DreamBooth (personalising models to a specific subject from 3–5 images), and LoRA (fine-tuning models on consumer hardware with minimal compute). The model was running on RTX 3090s, Apple M1 chips, and eventually in-browser. The community shipped more innovations in six months than the original team had produced in years.

The release triggered a legal and ethical reckoning. Artists whose work had been in the training data objected to their style being reproducible without credit or compensation. Getty Images sued Stability AI in January 2023. A group of artists filed a class action. The question of what "training on" someone's work means legally, and whether generating an image in someone's style constitutes infringement, had no clear answer in 2022. It still does not.

By 2024, latent diffusion had become the dominant architecture for image generation. Stable Diffusion XL, SDXL-Turbo, and FLUX.1 extended the original line. Text-to-video systems like Sora extended the same principles to temporal sequences. The model trained on images collected without explicit permission became the foundation of a multi-billion-dollar industry and one of the most contested legal questions in intellectual property law. Rombach's paper had been a PhD thesis chapter. It became the basis for nearly every image generator in use today.