IntermediateGenerative Networks

Generative Adversarial Networks (GAN)

A generative framework of two competing networks: a generator that turns random noise into samples, and a discriminator that tries to tell real samples from generated ones. The generator improves until the discriminator finds it hard to tell them apart.

Overview

NameGenerative Adversarial Networks
Year introduced2014
ResearchersIan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio
InstitutionUniversité de Montréal
Original paperGenerative adversarial nets arXiv
Data typeImages, Tabular data
DomainsGenerative AI, Computer vision, Medical image analysis
Computational costTwo passes per step (G and D), and inference is one generator pass.
Data sizeThousands to millions of samples depending on resolution.

Problem it solves

Explicit generative models needed probability calculations or complex approximations (Markov chains, lower bounds). What was wanted was a way to learn to generate realistic samples without computing the density explicitly.

Background and evolution

What came before: Multilayer Perceptron, Autoencoder

  1. 2014

    Goodfellow and colleagues introduced the GAN framework. [ref]

  2. 2016

    DCGAN set out stable convolutional architecture guidelines. [ref]

  3. 2017

    WGAN proposed the Wasserstein distance to improve stability. [ref]

  4. 2019

    StyleGAN raised the quality of face generation with a style-based architecture. [ref]

Core idea

The generator $G$ takes $z\sim\mathcal N(0,I)$ and outputs $G(z)$. The discriminator $D$ gives the probability that its input is real. They train alternately in a zero-sum game: $D$ increases its discrimination accuracy and $G$ tries to fool it. In practice a "non-saturating" objective is used for the generator because the original objective gives weak gradients at the start.

Example: Generating synthetic skin images to augment a rare class in the training data, while checking that the images do not copy real patients' images.

Interactive diagram

A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.

Select a layer in the diagram.

Components

ComponentRole
Generator GNoise → sample
Discriminator DSample → real/fake
Adversarial lossMinimax or non-saturating
Alternating optimizationA step for D, then for G

Equations

1. Minimax objective

$$\min_G\max_D\;V(D,G)=\mathbb{E}_{x\sim p_{data}}[\log D(x)]+\mathbb{E}_{z\sim p_z}[\log(1-D(G(z)))]$$

Symbols
$D(x)$Probability that x is real
$p_z$Noise distribution
Dimensions
A scalar
Numerical example
If $D(x)=0.9$ and $D(G(z))=0.2$, then $V=\ln0.9+\ln0.8\approx-0.105-0.223=-0.328$.
Relation to the architecture
D wants to maximise V and G wants to minimise it.

2. The optimal discriminator and the link to JSD

$$D^{*}(x)=\frac{p_{data}(x)}{p_{data}(x)+p_g(x)},\qquad V(D^{*},G)=-\log4+2\,\mathrm{JSD}(p_{data}\,\|\,p_g)$$

Symbols
$p_g$Generator distribution
$\mathrm{JSD}$Jensen–Shannon divergence
Dimensions
A scalar
Numerical example
When they match, $p_g=p_{data}$: $D^*=0.5$ and $V=-\log4\approx-1.386$.
Relation to the architecture
It shows that the optimal solution is matching the data distribution.

3. Non-saturating generator objective

$$\mathcal{L}_G=-\mathbb{E}_{z}\left[\log D(G(z))\right]$$

Symbols
$G$The generator
$D$The discriminator (probability that the sample is real)
$z$A noise vector from the prior distribution
Dimensions
A scalar
Numerical example
$D(G(z))=0.01$: the original objective gives $\ln(0.99)\approx-0.01$ (a weak gradient) while $-\ln(0.01)\approx4.6$ (a strong signal).
Relation to the architecture
The form proposed in the same paper for more stable training.

How it works, step by step

  1. Draw a real batch and a noise batch.
  2. Generate fake samples.
  3. Update D to tell real from fake.
  4. Update G so that D classifies the fake as real.
  5. Repeat, monitoring quality and diversity.

Strengths and limitations

Strengths

  • Sharp, realistic samples.
  • Fast inference with a single forward pass.
  • Needs no explicit density.

Limitations

  • Unstable training that is sensitive to hyperparameters.
  • Mode collapse.
  • Gives no explicit probability for a sample.
  • Hard to evaluate.

When to use it, and when not?

Good for

  • Fast generation of realistic samples.
  • Careful data augmentation.

Not suited for

  • When probabilities must be estimated.
  • When full coverage of diversity is essential.

Applications

DomainUse
Generative AIImage generation and super-resolution.
Computer visionImage-to-image translation.
Medical image analysisAugmenting rare medical data (with care about privacy and bias).

PyTorch code

A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.

Evaluation here uses two simple metrics computed inside the example: the share of samples close to one of the eight modes, and the number of modes covered. FID is not used because it is specific to images.

Python (PyTorch)
# GAN — a generator and a discriminator on a ring-shaped Gaussian mixture
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F

torch.manual_seed(0)

# 2) Define the model
def mlp(i, o, h=128):
    return nn.Sequential(nn.Linear(i, h), nn.LeakyReLU(0.2), nn.Linear(h, h), nn.LeakyReLU(0.2), nn.Linear(h, o))

class Generator(nn.Module):              # G: z ~ N(0, I) → x
    def __init__(self, z_dim=8):
        super().__init__(); self.z_dim, self.net = z_dim, mlp(z_dim, 2)
    def forward(self, z):
        return self.net(z)

class Discriminator(nn.Module):          # D: x → logit(real)
    def __init__(self):
        super().__init__(); self.net = mlp(2, 1)
    def forward(self, x):
        return self.net(x).squeeze(-1)

# 3) Prepare the data
# Target distribution: a mixture of 8 Gaussians on a circle of radius 2 (a classic test of mode collapse)
CENTERS = torch.stack([torch.tensor([2 * torch.cos(torch.tensor(k * torch.pi / 4)), 2 * torch.sin(torch.tensor(k * torch.pi / 4))]) for k in range(8)])
def sample_ring(n, std=0.1):
    k = torch.randint(0, 8, (n,))
    return CENTERS[k] + std * torch.randn(n, 2), k

def ring_metrics(x):
    """Quality: share of samples close (<0.3) to one of the centres. Coverage: number of modes that received ≥5% of the good samples."""
    d = torch.cdist(x, CENTERS)
    near, k = d.min(1)
    good = near < 0.3
    counts = torch.bincount(k[good], minlength=8).float()
    covered = int((counts >= 0.05 * max(1, good.sum().item())).sum())
    return good.float().mean().item(), covered

# 4) Loss function (non-saturating GAN loss) and the two optimizers
G, Dnet = Generator(), Discriminator()
bce = nn.BCEWithLogitsLoss()
opt_g = torch.optim.Adam(G.parameters(), lr=1e-3, betas=(0.5, 0.999))
opt_d = torch.optim.Adam(Dnet.parameters(), lr=1e-3, betas=(0.5, 0.999))

# 5) Training loop: a step for the discriminator, then a step for the generator
STEPS, BATCH = 3000, 256
for step in range(STEPS):
    real, _ = sample_ring(BATCH)
    fake = G(torch.randn(BATCH, G.z_dim))                                    # generator forward pass
    d_loss = bce(Dnet(real), torch.ones(BATCH)) + bce(Dnet(fake.detach()), torch.zeros(BATCH))
    opt_d.zero_grad(); d_loss.backward(); opt_d.step()
    g_loss = bce(Dnet(fake), torch.ones(BATCH))                              # the generator wants to be classified "real"
    opt_g.zero_grad(); g_loss.backward(); opt_g.step()
    if step % 750 == 0 or step == STEPS - 1:
        print(f"step {step:4d}  D_loss={d_loss.item():.3f}  G_loss={g_loss.item():.3f}")

# 6) Evaluation: sample quality and coverage of the eight modes
with torch.no_grad():
    q, cov = ring_metrics(G(torch.randn(4000, G.z_dim)))
print(f"sample quality (near a mode) = {q:.3f} ; modes covered = {cov}/8")

# 7) Inference: generate a new sample
with torch.no_grad():
    print("generated point:", [round(v, 3) for v in G(torch.randn(1, G.z_dim))[0].tolist()])

This code was run on 2026-11-20 in 12.7 seconds (PyTorch (CPU), Python 3 – local build machine).

Sample output:

sample quality (near a mode) = 0.827 ; modes covered = 6/8

Trainable parameters in the example: G = 17,922, Dnet = 17,025.

These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.

Try it and go deeper

Related architectures

Intermediate2014

Conditional Adversarial Networks

Conditional GAN (cGAN)

An extension of GANs in which a condition y (a class, text or image) is passed to both the generator and the discriminator, so the model learns the conditional distribut…

Generative Networks

Intermediate2017

CycleGAN

CycleGAN

Translation between two image domains without paired examples, using two opposite generators and two discriminators, with a "cycle consistency" loss that requires a roun…

Generative Networks

Advanced2019

StyleGAN

StyleGAN

A GAN generator that first transforms the noise through a mapping network into an intermediate latent space w, then injects "style" into every layer of a network that st…

Generative Networks

Intermediate2014

Variational Autoencoder

Variational Autoencoder (VAE)

A probabilistic generative model with a latent variable: an encoder that estimates an approximate distribution q(z x) and a decoder that represents p(x z), trained toget…

Generative Networks

Advanced2020

Diffusion Models

Diffusion Models (DDPM)

They gradually corrupt the data by adding Gaussian noise over T steps, then a network learns to reverse this process by predicting the added noise, so a new example is g…

Generative Networks

Research ideas

  • Evaluate GANs against diffusion models for augmenting rare skin-lesion image data, with a privacy leakage (memorisation) test.

Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.

Test your understanding

  1. Concept

    What is the optimal discriminator's value when the generator distribution matches the data distribution?

  2. Concept

    What is meant by mode collapse?

  3. Mathematical

    Why is the objective $-\log D(G(z))$ preferred for the generator?

References

  1. Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. Advances in Neural Information Processing Systems (NeurIPS) 27. https://arxiv.org/abs/1406.2661

    arXivAutomatically verified via OpenAlex/arXiv – 2026-11-20

  2. Radford, A., Metz, L., & Chintala, S. (2016). Unsupervised representation learning with deep convolutional generative adversarial networks. International Conference on Learning Representations (ICLR). https://arxiv.org/abs/1511.06434

    arXivAutomatically verified via OpenAlex/arXiv – 2026-11-20

  3. Arjovsky, M., Chintala, S., & Bottou, L. (2017). Wasserstein generative adversarial networks. International Conference on Machine Learning (ICML), PMLR 70, 214–223. https://arxiv.org/abs/1701.07875

    arXivAutomatically verified via OpenAlex/arXiv – 2026-11-20

  4. Mirza, M., & Osindero, S. (2014). Conditional generative adversarial nets. arXiv preprint. https://arxiv.org/abs/1411.1784

    arXivAutomatically verified via OpenAlex/arXiv – 2026-11-20

  5. Karras, T., Laine, S., & Aila, T. (2019). A style-based generator architecture for generative adversarial networks. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 4396–4405. https://doi.org/10.1109/CVPR.2019.00453

    DOIarXivAutomatically verified via Crossref – 2026-11-20