Kolmogorov–Arnold Networks
Kolmogorov–Arnold Networks (KAN)
Networks that place learnable activation functions (B-spline curves) on the edges instead of fixed weights, while the nodes only sum their inputs, inspired by the Kolmog…
The architectures you added to your favourites. They are stored in your browser only (LocalStorage) and are not sent to the site.
Kolmogorov–Arnold Networks (KAN)
Networks that place learnable activation functions (B-spline curves) on the edges instead of fixed weights, while the nodes only sum their inputs, inspired by the Kolmog…
Capsule Networks (CapsNet)
Networks that represent entities as capsules (vectors) whose length expresses the probability that the entity is present and whose direction expresses its properties (po…
Hybrid Capsule Networks (CNN-CapsNet)
Designs that use a strong convolutional backbone (such as ResNet) to extract features, then a capsule layer with dynamic routing on top to represent part–whole relations…
Spiking Neural Networks (SNN)
Networks whose neurons communicate with discrete binary spikes in time, like the brain. The membrane potential accumulates and fires a spike when it exceeds the threshol…
Liquid Neural Networks (LNN)
Continuous-time recurrent networks in which each neuron's time constant changes with the input (Liquid Time-Constant), so their dynamics adapt to the data with a small n…
Hypernetworks
A small network (the hypernetwork) generates the weights of another network (the target network), so the weights become a function of a conditioning input such as the ta…
Deep Equilibrium Models (DEQ)
Instead of stacking L layers, the model looks for a fixed point of a single layer z^*=f_θ(z^*,x), as if it were an infinitely deep network with shared weights, and compu…
Reservoir Computing (ESN / LSM)
A large recurrent network with fixed random weights (the reservoir) turns the input into rich states, and only the linear readout layer is trained, usually with closed-f…
Multilayer Perceptron (MLP)
A network of successive linear layers separated by nonlinear activation functions; it is the first building block on which most modern architectures are built.
Convolutional Neural Network (CNN)
A network that uses small convolution filters with shared weights sliding over the image, so it captures local patterns with far fewer parameters than an MLP.
Residual Network (ResNet)
It adds skip connections that make each block learn the residual difference F(x) instead of the full transformation, so training networks with hundreds of layers becomes…
Autoencoder (AE)
A network that compresses the input into a small latent representation (the encoder) and then reconstructs it (the decoder), so it learns a compressed representation wit…
Attention Mechanism
A mechanism that lets the model compute a weight for each input element according to its relevance to the current step, then take a weighted average, instead of compress…
U-Net
A U-shaped segmentation architecture: a contracting path that captures context and an expanding path that restores spatial resolution, with skip connections that carry f…
Attention U-Net
A U-Net with attention gates on the skip connections that learn to highlight the relevant regions and suppress the background before the features are merged.
Generative Adversarial Networks (GAN)
A generative framework of two competing networks: a generator that turns random noise into samples, and a discriminator that tries to tell real samples from generated on…
Conditional GAN (cGAN)
An extension of GANs in which a condition y (a class, text or image) is passed to both the generator and the discriminator, so the model learns the conditional distribut…
CycleGAN
Translation between two image domains without paired examples, using two opposite generators and two discriminators, with a "cycle consistency" loss that requires a roun…
StyleGAN
A GAN generator that first transforms the noise through a mapping network into an intermediate latent space w, then injects "style" into every layer of a network that st…
Variational Autoencoder (VAE)
A probabilistic generative model with a latent variable: an encoder that estimates an approximate distribution q(z x) and a decoder that represents p(x z), trained toget…
Vector Quantized Variational Autoencoder (VQ-VAE)
An autoencoder that quantises the encoder's output to the nearest vector in a learned codebook, so the representation becomes discrete codes; an autoregressive prior is…
Diffusion Models (DDPM)
They gradually corrupt the data by adding Gaussian noise over T steps, then a network learns to reverse this process by predicting the added noise, so a new example is g…
Latent Diffusion Models (LDM)
They apply diffusion in a compressed latent space from a pretrained autoencoder instead of pixel space, with a cross-attention mechanism for feeding in conditions (text,…
Score-Based Generative Models (SGM)
They learn a "score" network _x p(x) at multiple noise levels, then generate samples with Langevin dynamics that follow the score from noise towards high-density regions…
Flow Matching (FM)
It learns the velocity field of an ordinary differential equation that carries the noise distribution to the data distribution, by regressing directly on the velocities…
Graph Neural Network (GNN)
A family of models that learn a representation for each node by exchanging messages with its neighbours along the edges, combining node features with graph structure.
Graph Convolutional Network (GCN)
A simple graph layer that takes a normalised average of the representations of a node and its neighbours and then applies a shared linear transformation; it is a first-o…
Graph Attention Network (GAT)
It replaces GCN's fixed weights with learned attention coefficients for each edge, so each node decides how important each neighbour is, with multiple attention heads.
GraphSAGE
An inductive framework that learns aggregation functions over a sampled neighbourhood, so it can compute representations for new nodes the network did not see during tra…
Graph Isomorphism Network (GIN)
A GNN designed to reach the discriminative power of the Weisfeiler–Lehman test (1-WL), using sum aggregation and an MLP in the update.
Relational Graph Convolutional Network (R-GCN)
An extension of GCN for multi-relational graphs (such as knowledge graphs) that uses a different weight for each relation type, with a weight decomposition to reduce par…
Graph Transformer (GT)
Applying self-attention to the nodes of a graph with positional or structural encodings (such as Laplacian eigenvectors) so the model knows the graph's structure, going…
Temporal Graph Networks (TGN)
A framework for learning representations on dynamic graphs represented as a stream of timed events, using a memory for each node that is updated by events and an embeddi…
CNN-Transformer Hybrids
Designs that combine convolution (local bias, efficiency, generalisation with less data) with attention (global context, large capacity): usually a convolutional stem or…
CNN-LSTM
A CNN extracts a representation of each spatial element (a video frame, a signal window, an image), then an LSTM models the sequence of these representations over time;…
CNN-GRU
One-dimensional convolutions capture local patterns in the signal and shorten it, then a GRU (often bidirectional) models relationships over time with fewer parameters t…
U-Net + Transformer (U-Net+Transformer)
A family of segmentation models that keeps the U-Net shape (an encoder–decoder with skip connections) and introduces the transformer somewhere: the bottleneck, the whole…
TransUNet
A medical segmentation model that uses a hybrid encoder: a CNN (ResNet-50) extracts a feature map, which is split into patches and passed through a ViT, followed by a ca…
UNETR
A 3D segmentation model whose encoder is a pure ViT on 16³ volumetric patches; representations are taken from its layers (3, 6, 9, 12), reshaped and upsampled with trans…
Swin UNETR (Swin-UNETR)
A volumetric segmentation model whose encoder is a hierarchical Swin Transformer (shifted local windows and patch merging) that gives features at five scales, connected…
Attention-based CNN (Attention CNN)
Convolutional networks that add lightweight attention modules that reweight the channels (which features matter most) or the positions (where the model looks), at a very…
Multi-Branch Neural Networks
Architectures that process the same input or different inputs through parallel branches (different kernel sizes, different modalities or different scales) and then merge…
Physics-Informed Neural Networks (PINN)
A network that approximates the solution of a partial differential equation u_θ(x,t) and is trained with a loss that combines the equation's residual (computed by automa…
Deep Operator Network (DeepONet)
A network that learns an operator mapping a function to a function (such as: condition → solution), through a "branch" network that encodes the input function from its v…
Fourier Neural Operator (FNO)
A neural operator that applies a "spectral convolution" in each layer: a Fourier transform of the features, then multiplication of a limited number of low frequencies by…
Neural Operators (NO)
A family of architectures that learn operators between function spaces rather than between fixed-size vectors, by stacking pointwise linear layers with integral operator…
Physics-Informed Deep Learning (PIML)
A general approach to embedding physical knowledge in deep learning in three ways: through data (observational bias), through the architecture (inductive bias, such as H…
Scientific Machine Learning Architectures (SciML)
A family of architectures that blends mechanistic models (known equations) with neural networks that learn the unknown parts, trained through differentiable solvers. Not…
Recurrent Neural Network (RNN)
A network that processes a sequence element by element and carries a hidden state that passes information from previous steps, using the same weights at every step.
Long Short-Term Memory (LSTM)
A recurrent network with a memory cell and gates (forget, input, output) that control what is kept and what is erased, so it learns long-term dependencies far better tha…
Gated Recurrent Unit (GRU)
A simplification of LSTM with only two gates (update and reset) and no separate cell state, with comparable performance on many tasks and fewer parameters.
Bidirectional LSTM (BiLSTM)
Two recurrent layers, one reading the sequence from the start and the other from the end; their states are concatenated so that every position sees both its preceding an…
Temporal Convolutional Network (TCN)
A causal 1D convolutional network (it cannot see the future) with dilated convolutions and residual connections, which processes sequences in parallel and covers a long…
Neural Ordinary Differential Equations (Neural ODE)
The hidden state is defined by a continuous differential equation whose derivative is given by a neural network, and the output is computed with a numerical solver, so t…
Structured State Space Models (SSM / S4)
Sequence layers built on the continuous linear state space model, discretized in time and computable as a fast recurrence at inference or as a long parallel convolution…
Mamba (Selective State Space Models)
An SSM in which B, C and the step Δ are functions of the input (selective), so the layer decides what to remember and what to ignore based on content, with a hardware-aw…
Mixture of Experts (MoE)
Several "expert" networks and a gating network that decides, for each input, the weight of each expert in the output. The experts learn to specialize in different region…
Sparse Mixture of Experts (Sparse MoE)
An expert layer in which only a small number of experts (top-k) is activated per token, so the model's parameter count grows greatly without a similar increase in per-to…
Modular Neural Networks (Modular NN)
Networks built from reusable modules (such as Find, And and Count) composed dynamically according to the structure of the task or question, so skills transfer to new com…
Neural Architecture Search (NAS)
Automating architecture design: a search space that defines possible architectures, a search strategy (reinforcement learning, evolution, Bayesian, gradient) that propos…
Differentiable Architecture Search (DARTS)
Turns architecture search into a continuous optimization problem: each edge in the cell computes a weighted mixture of all possible operations with Softmax weights over…
Equivariant Neural Networks (ENN)
Networks built so that their output transforms in a specific way when the input is transformed by a symmetry (rotation, translation, reflection, permutation), so they do…
Energy-Based Models (EBM)
Models that learn a scalar energy function E_θ(x) that is low for likely configurations and high for others, and define the density implicitly p(x) e^-E(x) without compu…
Retrieval-Augmented Neural Architectures (RAG)
Models that combine a non-parametric memory (a document index) with a parametric model: a retriever finds passages relevant to the question, and a generator uses them to…
Transformer
An architecture based entirely on multi-head self-attention instead of recurrence or convolution, so it processes all sequence positions in parallel and links any two po…
Vision Transformer (ViT)
Applies the standard Transformer encoder directly to images after splitting them into fixed-size patches treated as tokens, with a special classification token and a lea…
Swin Transformer
A hierarchical vision transformer that computes attention within non-overlapping local windows and shifts the windows in successive layers to exchange information betwee…
Data-efficient Image Transformers (DeiT)
A method for training ViT on ImageNet-1k alone with strong data augmentation and regularization, plus an extra "distillation token" that learns from the predictions of a…
Perceiver
A general architecture that reads a huge input of any type through cross-attention into a small fixed-size latent array, then processes the latents with self-attention,…
Cross-Attention (Cross-Attn)
Attention in which the queries come from one sequence while the keys and values come from another sequence or modality, so each element of the first reads what it needs…
Multimodal Transformers (MM-Transformer)
Transformers that process more than one modality (image, text, audio...) either in a single sequence with a modality-type embedding (single-stream), or in two streams th…
Vision-Language Models (VLM)
Models that connect a visual encoder to a language model: either through contrastive alignment in a shared space (CLIP), or by turning visual features into tokens that a…
No results match the filters.
You have not added any architecture yet. Press the heart button on a card or an architecture page.
Sections
No section has this name. Press “Search” to search all site content.
Favourites
My results & certificates
My academy progressMy quiz resultsKids’ progressAcademy certificatesQuiz certificatesStart any lesson or quiz and it will show up here so you can get back to it quickly.
Settings
I am the smart assistant for deep learning architectures. Ask me about an architecture, an equation or code, or ask for a comparison, a path or an explanation of a lab result.
Answers from the encyclopedia's content only, with no language model and without sending your question to any outside party.
The assistant can make mistakes, so check the attached sources.