Generative Adversarial Network (GAN)

Why it exists

Many tasks need a model that can generate new realistic data, faces, textures, audio, rather than just classify existing data. Earlier generative models required specifying and optimizing an explicit probability density, which is hard for complex high-dimensional data like images. GANs sidestep that by learning to generate through competition instead of likelihood maximization.

The mental model

Think of a counterfeiter and a detective. The generator is the counterfeiter producing fake currency; the discriminator is the detective trying to spot fakes. As the detective gets sharper, the counterfeiter must produce better fakes, and vice versa. This arms race continues until the counterfeits are good enough that the detective can do no better than guessing.

How it works

The generator maps a random noise vector to a synthetic sample. The discriminator receives a mix of real samples and generated ones and outputs the probability that each is real. Training is a minimax game: the discriminator maximizes its accuracy at telling real from fake, while the generator minimizes the discriminator's ability to detect its fakes. Gradients from the discriminator flow back to teach the generator. At the theoretical optimum the generator's distribution matches the real data distribution and the discriminator outputs one half everywhere.

When it matters

GANs shine for high-fidelity image synthesis, style transfer, super-resolution, and augmenting scarce datasets. They matter less where you need exact likelihoods or stable, easily trained generation, where diffusion models or variational autoencoders may be preferable. Key failure modes to know are training instability and mode collapse, where the generator produces a narrow set of outputs that fool the discriminator without covering the full data variety.

A concrete example

To generate realistic human faces, you feed the generator random noise vectors and it outputs face images; the discriminator is shown both real photographs and these generated faces and learns to flag fakes. Early on the faces are blurry blobs the discriminator easily rejects. Over many rounds the generator learns texture, symmetry, and lighting to evade detection, eventually producing convincing faces of people who do not exist, the result popularized by sites that display GAN-generated portraits.