Foundations

The architectures the rest of the encyclopedia builds on: the multilayer perceptron, convolution, residual connections, the autoencoder, the attention mechanism and U-Net. They are not among the specification's eight categories, but they are a prerequisite for most learning paths.

Beginner1986

Multilayer Perceptron

Multilayer Perceptron (MLP)

A network of successive linear layers separated by nonlinear activation functions; it is the first building block on which most modern architectures are built.

Foundations

Beginner1998

Convolutional Neural Network

Convolutional Neural Network (CNN)

A network that uses small convolution filters with shared weights sliding over the image, so it captures local patterns with far fewer parameters than an MLP.

Foundations

Basic2016

Residual Network

Residual Network (ResNet)

It adds skip connections that make each block learn the residual difference F(x) instead of the full transformation, so training networks with hundreds of layers becomes…

Foundations

Beginner2006

Autoencoder

Autoencoder (AE)

A network that compresses the input into a small latent representation (the encoder) and then reconstructs it (the decoder), so it learns a compressed representation wit…

FoundationsGenerative Networks

Basic2015

Attention Mechanism

Attention Mechanism

A mechanism that lets the model compute a weight for each input element according to its relevance to the current step, then take a weighted average, instead of compress…

FoundationsTemporal and Sequential Networks

Basic2015

U-Net

U-Net

A U-shaped segmentation architecture: a contracting path that captures context and an expanding path that restores spatial resolution, with skip connections that carry f…

FoundationsHybrid Architectures