My favourites

The architectures you added to your favourites. They are stored in your browser only (LocalStorage) and are not sent to the site.

Advanced2024

Kolmogorov–Arnold Networks

Kolmogorov–Arnold Networks (KAN)

Networks that place learnable activation functions (B-spline curves) on the edges instead of fixed weights, while the nodes only sum their inputs, inspired by the Kolmog…

Advanced Neural Architectures

Intermediate2017

Capsule Networks

Capsule Networks (CapsNet)

Networks that represent entities as capsules (vectors) whose length expresses the probability that the entity is present and whose direction expresses its properties (po…

Advanced Neural Architectures

Intermediate2017

Hybrid Capsule Networks

Hybrid Capsule Networks (CNN-CapsNet)

Designs that use a strong convolutional backbone (such as ResNet) to extract features, then a capsule layer with dynamic routing on top to represent part–whole relations…

Hybrid Architectures

Intermediate1997

Spiking Neural Networks

Spiking Neural Networks (SNN)

Networks whose neurons communicate with discrete binary spikes in time, like the brain. The membrane potential accumulates and fires a spike when it exceeds the threshol…

Advanced Neural Architectures

Advanced2021

Liquid Neural Networks

Liquid Neural Networks (LNN)

Continuous-time recurrent networks in which each neuron's time constant changes with the input (Liquid Time-Constant), so their dynamics adapt to the data with a small n…

Temporal and Sequential NetworksAdvanced Neural Architectures

Intermediate2017

Hypernetworks

Hypernetworks

A small network (the hypernetwork) generates the weights of another network (the target network), so the weights become a function of a conditioning input such as the ta…

Advanced Neural ArchitecturesSpecialized Architectures

Advanced2019

Deep Equilibrium Models

Deep Equilibrium Models (DEQ)

Instead of stacking L layers, the model looks for a fixed point of a single layer z^*=f_θ(z^*,x), as if it were an infinitely deep network with shared weights, and compu…

Advanced Neural Architectures

Basic2004

Reservoir Computing

Reservoir Computing (ESN / LSM)

A large recurrent network with fixed random weights (the reservoir) turns the input into rich states, and only the linear readout layer is trained, usually with closed-f…

Temporal and Sequential NetworksAdvanced Neural Architectures

Beginner1986

Multilayer Perceptron

Multilayer Perceptron (MLP)

A network of successive linear layers separated by nonlinear activation functions; it is the first building block on which most modern architectures are built.

Foundations

Beginner1998

Convolutional Neural Network

Convolutional Neural Network (CNN)

A network that uses small convolution filters with shared weights sliding over the image, so it captures local patterns with far fewer parameters than an MLP.

Foundations

Basic2016

Residual Network

Residual Network (ResNet)

It adds skip connections that make each block learn the residual difference F(x) instead of the full transformation, so training networks with hundreds of layers becomes…

Foundations

Beginner2006

Autoencoder

Autoencoder (AE)

A network that compresses the input into a small latent representation (the encoder) and then reconstructs it (the decoder), so it learns a compressed representation wit…

FoundationsGenerative Networks

Basic2015

Attention Mechanism

Attention Mechanism

A mechanism that lets the model compute a weight for each input element according to its relevance to the current step, then take a weighted average, instead of compress…

FoundationsTemporal and Sequential Networks

Basic2015

U-Net

U-Net

A U-shaped segmentation architecture: a contracting path that captures context and an expanding path that restores spatial resolution, with skip connections that carry f…

FoundationsHybrid Architectures

Basic2018

Attention U-Net

Attention U-Net

A U-Net with attention gates on the skip connections that learn to highlight the relevant regions and suppress the background before the features are merged.

Hybrid Architectures

Intermediate2014

Generative Adversarial Networks

Generative Adversarial Networks (GAN)

A generative framework of two competing networks: a generator that turns random noise into samples, and a discriminator that tries to tell real samples from generated on…

Generative Networks

Intermediate2014

Conditional Adversarial Networks

Conditional GAN (cGAN)

An extension of GANs in which a condition y (a class, text or image) is passed to both the generator and the discriminator, so the model learns the conditional distribut…

Generative Networks

Intermediate2017

CycleGAN

CycleGAN

Translation between two image domains without paired examples, using two opposite generators and two discriminators, with a "cycle consistency" loss that requires a roun…

Generative Networks

Advanced2019

StyleGAN

StyleGAN

A GAN generator that first transforms the noise through a mapping network into an intermediate latent space w, then injects "style" into every layer of a network that st…

Generative Networks

Intermediate2014

Variational Autoencoder

Variational Autoencoder (VAE)

A probabilistic generative model with a latent variable: an encoder that estimates an approximate distribution q(z x) and a decoder that represents p(x z), trained toget…

Generative Networks

Advanced2017

Vector-Quantised Variational Autoencoder

Vector Quantized Variational Autoencoder (VQ-VAE)

An autoencoder that quantises the encoder's output to the nearest vector in a learned codebook, so the representation becomes discrete codes; an autoregressive prior is…

Generative Networks

Advanced2020

Diffusion Models

Diffusion Models (DDPM)

They gradually corrupt the data by adding Gaussian noise over T steps, then a network learns to reverse this process by predicting the added noise, so a new example is g…

Generative Networks

Advanced2022

Latent Diffusion Models

Latent Diffusion Models (LDM)

They apply diffusion in a compressed latent space from a pretrained autoencoder instead of pixel space, with a cross-attention mechanism for feeding in conditions (text,…

Generative NetworksHybrid Architectures

Research2019

Score-Based Generative Models

Score-Based Generative Models (SGM)

They learn a "score" network _x p(x) at multiple noise levels, then generate samples with Langevin dynamics that follow the score from noise towards high-density regions…

Generative Networks

Research2023

Flow Matching

Flow Matching (FM)

It learns the velocity field of an ordinary differential equation that carries the noise distribution to the data distribution, by regressing directly on the velocities…

Generative Networks

Basic2009

Graph Neural Network

Graph Neural Network (GNN)

A family of models that learn a representation for each node by exchanging messages with its neighbours along the edges, combining node features with graph structure.

Graph Neural Networks

Basic2017

Graph Convolutional Network

Graph Convolutional Network (GCN)

A simple graph layer that takes a normalised average of the representations of a node and its neighbours and then applies a shared linear transformation; it is a first-o…

Graph Neural Networks

Intermediate2018

Graph Attention Network

Graph Attention Network (GAT)

It replaces GCN's fixed weights with learned attention coefficients for each edge, so each node decides how important each neighbour is, with multiple attention heads.

Graph Neural Networks

Basic2017

GraphSAGE

GraphSAGE

An inductive framework that learns aggregation functions over a sampled neighbourhood, so it can compute representations for new nodes the network did not see during tra…

Graph Neural Networks

Intermediate2019

Graph Isomorphism Network

Graph Isomorphism Network (GIN)

A GNN designed to reach the discriminative power of the Weisfeiler–Lehman test (1-WL), using sum aggregation and an MLP in the update.

Graph Neural Networks

Intermediate2018

Relational Graph Convolutional Network

Relational Graph Convolutional Network (R-GCN)

An extension of GCN for multi-relational graphs (such as knowledge graphs) that uses a different weight for each relation type, with a weight decomposition to reduce par…

Graph Neural Networks

Advanced2020

Graph Transformer

Graph Transformer (GT)

Applying self-attention to the nodes of a graph with positional or structural encodings (such as Laplacian eigenvectors) so the model knows the graph's structure, going…

Graph Neural NetworksSpecialized ArchitecturesTransformer Architectures

Advanced2020

Temporal Graph Networks

Temporal Graph Networks (TGN)

A framework for learning representations on dynamic graphs represented as a stream of timed events, using a memory for each node that is updated by events and an embeddi…

Temporal and Sequential NetworksGraph Neural Networks

Intermediate2021

Hybrid CNN–Transformer Networks

CNN-Transformer Hybrids

Designs that combine convolution (local bias, efficiency, generalisation with less data) with attention (global context, large capacity): usually a convolutional stem or…

Hybrid ArchitecturesTransformer Architectures

Basic2015

Hybrid CNN–LSTM Networks

CNN-LSTM

A CNN extracts a representation of each spatial element (a video frame, a signal window, an image), then an LSTM models the sequence of these representations over time;…

Temporal and Sequential NetworksHybrid Architectures

Basic2014

Hybrid CNN–GRU Networks

CNN-GRU

One-dimensional convolutions capture local patterns in the signal and shorten it, then a GRU (often bidirectional) models relationships over time with fewer parameters t…

Temporal and Sequential NetworksHybrid Architectures

Intermediate2021

U-Net with Transformers

U-Net + Transformer (U-Net+Transformer)

A family of segmentation models that keeps the U-Net shape (an encoder–decoder with skip connections) and introduces the transformer somewhere: the bottleneck, the whole…

Hybrid ArchitecturesTransformer Architectures

Intermediate2021

TransUNet

TransUNet

A medical segmentation model that uses a hybrid encoder: a CNN (ResNet-50) extracts a feature map, which is split into patches and passed through a ViT, followed by a ca…

Hybrid ArchitecturesTransformer Architectures

Advanced2022

UNETR

UNETR

A 3D segmentation model whose encoder is a pure ViT on 16³ volumetric patches; representations are taken from its layers (3, 6, 9, 12), reshaped and upsampled with trans…

Hybrid ArchitecturesTransformer Architectures

Advanced2022

Swin UNETR

Swin UNETR (Swin-UNETR)

A volumetric segmentation model whose encoder is a hierarchical Swin Transformer (shifted local windows and patch merging) that gives features at five scales, connected…

Hybrid ArchitecturesTransformer Architectures

Basic2018

Attention-Augmented Convolutional Networks

Attention-based CNN (Attention CNN)

Convolutional networks that add lightweight attention modules that reweight the channels (which features matter most) or the positions (where the model looks), at a very…

Hybrid Architectures

Basic2015

Multi-Branch Neural Networks

Multi-Branch Neural Networks

Architectures that process the same input or different inputs through parallel branches (different kernel sizes, different modalities or different scales) and then merge…

Hybrid Architectures

Intermediate2019

Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINN)

A network that approximates the solution of a partial differential equation u_θ(x,t) and is trained with a loss that combines the equation's residual (computed by automa…

Physics-Informed Neural Networks

Advanced2021

Deep Operator Network

Deep Operator Network (DeepONet)

A network that learns an operator mapping a function to a function (such as: condition → solution), through a "branch" network that encodes the input function from its v…

Physics-Informed Neural Networks

Advanced2021

Fourier Neural Operator

Fourier Neural Operator (FNO)

A neural operator that applies a "spectral convolution" in each layer: a Fourier transform of the features, then multiplication of a limited number of low frequencies by…

Physics-Informed Neural Networks

Research2023

Neural Operators

Neural Operators (NO)

A family of architectures that learn operators between function spaces rather than between fixed-size vectors, by stacking pointwise linear layers with integral operator…

Physics-Informed Neural Networks

Advanced2021

Physics-informed deep learning

Physics-Informed Deep Learning (PIML)

A general approach to embedding physical knowledge in deep learning in three ways: through data (observational bias), through the architecture (inductive bias, such as H…

Physics-Informed Neural Networks

Advanced2020

Scientific machine learning architectures

Scientific Machine Learning Architectures (SciML)

A family of architectures that blends mechanistic models (known equations) with neural networks that learn the unknown parts, trained through differentiable solvers. Not…

Physics-Informed Neural NetworksHybrid Architectures

Beginner1990

Recurrent neural network

Recurrent Neural Network (RNN)

A network that processes a sequence element by element and carries a hidden state that passes information from previous steps, using the same weights at every step.

Temporal and Sequential Networks

Basic1997

Long short-term memory

Long Short-Term Memory (LSTM)

A recurrent network with a memory cell and gates (forget, input, output) that control what is kept and what is erased, so it learns long-term dependencies far better tha…

Temporal and Sequential Networks

Basic2014

Gated recurrent unit

Gated Recurrent Unit (GRU)

A simplification of LSTM with only two gates (update and reset) and no separate cell state, with comparable performance on many tasks and fewer parameters.

Temporal and Sequential Networks

Basic1997

Bidirectional LSTM

Bidirectional LSTM (BiLSTM)

Two recurrent layers, one reading the sequence from the start and the other from the end; their states are concatenated so that every position sees both its preceding an…

Temporal and Sequential Networks

Basic2018

Temporal convolutional networks

Temporal Convolutional Network (TCN)

A causal 1D convolutional network (it cannot see the future) with dilated convolutions and residual connections, which processes sequences in parallel and covers a long…

Temporal and Sequential Networks

Advanced2018

Neural ordinary differential equations

Neural Ordinary Differential Equations (Neural ODE)

The hidden state is defined by a continuous differential equation whose derivative is given by a neural network, and the output is computed with a numerical solver, so t…

Temporal and Sequential NetworksPhysics-Informed Neural Networks

Advanced2022

State space models

Structured State Space Models (SSM / S4)

Sequence layers built on the continuous linear state space model, discretized in time and computable as a fast recurrence at inference or as a long parallel convolution…

Temporal and Sequential Networks

Advanced2023

Mamba: selective state space models

Mamba (Selective State Space Models)

An SSM in which B, C and the step Δ are functions of the input (selective), so the layer decides what to remember and what to ignore based on content, with a hardware-aw…

Temporal and Sequential Networks

Intermediate1991

Mixture of experts

Mixture of Experts (MoE)

Several "expert" networks and a gating network that decides, for each input, the weight of each expert in the output. The experts learn to specialize in different region…

Specialized Architectures

Advanced2017

Sparse mixture of experts

Sparse Mixture of Experts (Sparse MoE)

An expert layer in which only a small number of experts (top-k) is activated per token, so the model's parameter count grows greatly without a similar increase in per-to…

Specialized Architectures

Advanced2016

Modular neural networks

Modular Neural Networks (Modular NN)

Networks built from reusable modules (such as Find, And and Count) composed dynamically according to the structure of the task or question, so skills transfer to new com…

Specialized Architectures

Advanced2017

Neural architecture search

Neural Architecture Search (NAS)

Automating architecture design: a search space that defines possible architectures, a search strategy (reinforcement learning, evolution, Bayesian, gradient) that propos…

Specialized Architectures

Research2019

Differentiable architecture search

Differentiable Architecture Search (DARTS)

Turns architecture search into a continuous optimization problem: each edge in the cell computes a weighted mixture of all possible operations with Softmax weights over…

Specialized Architectures

Research2016

Equivariant neural networks

Equivariant Neural Networks (ENN)

Networks built so that their output transforms in a specific way when the input is transformed by a symmetry (rotation, translation, reflection, permutation), so they do…

Specialized Architectures

Advanced2006

Energy-based models

Energy-Based Models (EBM)

Models that learn a scalar energy function E_θ(x) that is low for likely configurations and high for others, and define the density implicitly p(x) e^-E(x) without compu…

Generative NetworksSpecialized Architectures

Advanced2020

Retrieval-augmented neural architectures

Retrieval-Augmented Neural Architectures (RAG)

Models that combine a non-parametric memory (a document index) with a parametric model: a retriever finds passages relevant to the question, and a generator uses them to…

Specialized ArchitecturesTransformer Architectures

Basic2017

Transformer

Transformer

An architecture based entirely on multi-head self-attention instead of recurrence or convolution, so it processes all sequence positions in parallel and links any two po…

Transformer Architectures

Intermediate2021

Vision Transformer

Vision Transformer (ViT)

Applies the standard Transformer encoder directly to images after splitting them into fixed-size patches treated as tokens, with a special classification token and a lea…

Transformer Architectures

Advanced2021

Swin Transformer

Swin Transformer

A hierarchical vision transformer that computes attention within non-overlapping local windows and shifts the windows in successive layers to exchange information betwee…

Transformer Architectures

Intermediate2021

Data-efficient image transformers

Data-efficient Image Transformers (DeiT)

A method for training ViT on ImageNet-1k alone with strong data augmentation and regularization, plus an extra "distillation token" that learns from the predictions of a…

Transformer Architectures

Advanced2021

Perceiver

Perceiver

A general architecture that reads a huge input of any type through cross-attention into a small fixed-size latent array, then processes the latents with self-attention,…

Transformer Architectures

Basic2017

Cross-attention

Cross-Attention (Cross-Attn)

Attention in which the queries come from one sequence while the keys and values come from another sequence or modality, so each element of the first reads what it needs…

Transformer Architectures

Intermediate2019

Multimodal transformers

Multimodal Transformers (MM-Transformer)

Transformers that process more than one modality (image, text, audio...) either in a single sequence with a modality-type embedding (single-stream), or in two streams th…

Transformer Architectures

Advanced2021

Vision-language models

Vision-Language Models (VLM)

Models that connect a visual encoder to a language model: either through contrastive alignment in a shared space (CLIP), or by turning visual features into tokens that a…

Transformer Architectures