IntermediateAdvanced Neural Architectures

Capsule Networks (CapsNet)

Networks that represent entities as capsules (vectors) whose length expresses the probability that the entity is present and whose direction expresses its properties (pose), and that link levels with a dynamic routing-by-agreement algorithm.

Overview

NameCapsule Networks
Year introduced2017
ResearchersSara Sabour, Nicholas Frosst, Geoffrey E. Hinton
InstitutionGoogle Brain
Original paperDynamic routing between capsules arXiv
Data typeImages
DomainsComputer vision, Medical image analysis
Computational costRelatively high, because of the predictions for every pair of capsules and the routing iterations.
Data sizeSmall and medium datasets (MNIST, smallNORB).

Problem it solves

Max pooling in a CNN loses information about the spatial relationships between parts, so the network may classify a face with scrambled parts as a face.

Background and evolution

What came before: Convolutional Neural Network

  1. 2017

    Sabour, Frosst and Hinton introduced dynamic routing between capsules at NeurIPS 2017. [ref]

  2. 2018

    Hinton and colleagues proposed matrix capsules with EM routing. [ref]

  3. 2022

    A comprehensive survey of capsule research and its challenges. [ref]

Core idea

Each capsule at the lower level (a part) predicts the pose of each higher capsule (a whole) through a transformation matrix. If the predictions of several parts agree on a similar pose, the coupling coefficients towards that capsule increase. The squash function keeps the vector length between 0 and 1.

Example: An "eye" capsule and a "mouth" capsule each predict the position and orientation of the "face"; if the two predictions agree, the face capsule is strongly activated, but if the parts are in the wrong places the predictions will not agree.

Interactive diagram

A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.

Select a layer in the diagram.

Components

ComponentRole
Conv layerExtracting low-level features
Primary CapsulesA convolution reshaped into small vectors
Transformation matrices$W_{ij}$ for the part's prediction of the whole
Dynamic routingRepeated updates of the coupling coefficients
SquashNormalising the vector length
Margin lossA loss on the lengths of the class capsules
Reconstruction decoderReconstructing the image as a regulariser

Equations

1. The squash function

$$v_j=\frac{\|s_j\|^2}{1+\|s_j\|^2}\,\frac{s_j}{\|s_j\|}$$

Symbols
$s_j$Total input of capsule j
$v_j$Its output
Dimensions
$v_j\in\mathbb{R}^{16}$ and $\|v_j\|<1$
Numerical example
If $\|s_j\|=1$ then $\|v_j\|=0.5$; if $\|s_j\|=3$ then $\|v_j\|=0.9$.
Relation to the architecture
It makes the length express the probability of presence.

2. Predictions and routing

$$\hat u_{j|i}=W_{ij}u_i,\quad s_j=\sum_i c_{ij}\hat u_{j|i},\quad c_{ij}=\frac{\exp b_{ij}}{\sum_k\exp b_{ik}},\quad b_{ij}\leftarrow b_{ij}+\hat u_{j|i}\cdot v_j$$

Symbols
$c_{ij}$Coupling coefficient
$b_{ij}$Initial logit starting at zero
Dimensions
$W_{ij}\in\mathbb{R}^{16\times8}$
Numerical example
If $\hat u_{j|i}\cdot v_j=2$ for one capsule and $-1$ for another, $b$ rises for the first, so its coupling increases in the next iteration.
Relation to the architecture
Agreement between the prediction and the output strengthens the link.

3. Margin loss

$$L_k=T_k\max(0,m^+-\|v_k\|)^2+\lambda(1-T_k)\max(0,\|v_k\|-m^-)^2$$

Symbols
$T_k$1 if class k is present
$m^+,m^-$0.9 and 0.1
$\lambda$0.5
Dimensions
A scalar for each class
Numerical example
A present class whose capsule length is $0.7$: $L=(0.9-0.7)^2=0.04$.
Relation to the architecture
It allows more than one class in the same image.

How it works, step by step

  1. An initial convolution extracts local features.
  2. The primary capsules are formed and squashed.
  3. Each capsule predicts the pose of the class capsules.
  4. Routing is repeated r times to update the coupling.
  5. The class is given by the longest capsule.
  6. A reconstruction decoder is added as a regulariser.

Strengths and limitations

Strengths

  • An explicit representation of part–whole relationships and pose.
  • Good results on MNIST and on overlapping digits (MultiMNIST) in the original paper.

Limitations

  • Iterative routing is costly and hard to scale to large images.
  • It has not shown a consistent advantage over modern CNNs on large image datasets.

When to use it, and when not?

Good for

  • Studying spatial relationships and pose.
  • Research experiments on small data.

Not suited for

  • High-resolution images and huge datasets, where the cost is large.

Applications

DomainUse
Computer visionRecognising overlapping digits and objects in different poses.
Medical image analysisExperiments in classifying small medical images.

PyTorch code

A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.

A miniature version for 16×16 images without the reconstruction decoder; the original paper uses MNIST with 28×28 images and 1152 primary capsules.

Python (PyTorch)
# Capsule Network with dynamic routing — a miniature version
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F

torch.manual_seed(0)

# 2) Define the model
def squash(s, dim=-1):
    n2 = (s ** 2).sum(dim, keepdim=True)
    return n2 / (1 + n2) * s / torch.sqrt(n2 + 1e-8)

class CapsNet(nn.Module):
    def __init__(self, n_classes=2, routing=3):
        super().__init__()
        self.conv = nn.Conv2d(1, 32, 5)                         # 16→12
        self.primary = nn.Conv2d(32, 8 * 8, 5, stride=2)        # 12→4 ⇒ 8 maps × 8D capsules
        self.n_in, self.r = 8 * 4 * 4, routing
        self.W = nn.Parameter(torch.randn(1, self.n_in, n_classes, 12, 8) * 0.05)
    def forward(self, x):
        h = F.relu(self.conv(x))
        u = self.primary(h).view(x.shape[0], 8, 8, 4, 4).permute(0, 1, 3, 4, 2).reshape(x.shape[0], -1, 8)
        u = squash(u)                                           # B×128×8
        u_hat = (self.W @ u[:, :, None, :, None]).squeeze(-1)   # B×128×C×12
        b = torch.zeros(u_hat.shape[:3])
        for it in range(self.r):                                # routing by agreement
            c = torch.softmax(b, dim=2)
            v = squash((c.unsqueeze(-1) * u_hat).sum(1))        # B×C×12
            if it < self.r - 1:
                b = b + (u_hat * v.unsqueeze(1)).sum(-1)
        return v.norm(dim=-1)                                   # length of each capsule

def margin_loss(lengths, y, m_pos=0.9, m_neg=0.1, lam=0.5):
    T = F.one_hot(y, lengths.shape[1]).float()
    L = T * F.relu(m_pos - lengths) ** 2 + lam * (1 - T) * F.relu(lengths - m_neg) ** 2
    return L.sum(1).mean()

# 3) Prepare the data
def make(n):
    X = torch.randn(n, 1, 16, 16) * 0.3
    y = torch.randint(0, 2, (n,))
    pos = torch.randint(2, 14, (n,))
    for i in range(n):
        if y[i] == 0: X[i, 0, pos[i], 2:14] += 1.5
        else:         X[i, 0, 2:14, pos[i]] += 1.5
    return X, y
X_train, y_train = make(384)
X_test, y_test = make(96)
model = CapsNet()

# 4) Margin loss and optimizer
opt = torch.optim.Adam(model.parameters(), lr=3e-3)

# 5) Training loop
for epoch in range(6):
    model.train()
    for i in range(0, len(X_train), 32):
        xb, yb = X_train[i:i + 32], y_train[i:i + 32]
        loss = margin_loss(model(xb), yb)        # Forward Pass + Margin Loss
        opt.zero_grad(); loss.backward(); opt.step()
    print(f"epoch {epoch}  loss={loss.item():.4f}")

# 6) Evaluation: class = longest capsule
model.eval()
with torch.no_grad():
    acc = (model(X_test).argmax(1) == y_test).float().mean().item()
print(f"test accuracy = {acc:.3f}")

# 7) Inference: class-capsule lengths for one image
with torch.no_grad():
    print("capsule lengths:", [round(v, 3) for v in model(X_test[:1])[0].tolist()])

This code was run on 2026-11-20 in 3.4 seconds (PyTorch (CPU), Python 3 – local build machine).

Sample output:

test accuracy = 1.000

Trainable parameters in the example: model = 76,672.

These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.

Try it and go deeper

Related architectures

Beginner1998

Convolutional Neural Network

Convolutional Neural Network (CNN)

A network that uses small convolution filters with shared weights sliding over the image, so it captures local patterns with far fewer parameters than an MLP.

Foundations

Intermediate2017

Hybrid Capsule Networks

Hybrid Capsule Networks (CNN-CapsNet)

Designs that use a strong convolutional backbone (such as ResNet) to extract features, then a capsule layer with dynamic routing on top to represent part–whole relations…

Hybrid Architectures

Basic2015

Attention Mechanism

Attention Mechanism

A mechanism that lets the model compute a weight for each input element according to its relevance to the current step, then take a weighted average, instead of compress…

FoundationsTemporal and Sequential Networks

Research ideas

  • Evaluate the robustness of CapsNet to rotation compared with a CNN given equivalent data augmentation.
  • Lightweight capsules for classifying small medical images, with a cost analysis.

Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.

Test your understanding

  1. Concept

    What does the length of a capsule vector represent in CapsNet?

  2. Concept

    What is the squashed length of a vector of length 1?

  3. Concept

    What is the idea behind dynamic routing?

References

  1. Sabour, S., Frosst, N., & Hinton, G. E. (2017). Dynamic routing between capsules. Advances in Neural Information Processing Systems (NeurIPS) 30. https://arxiv.org/abs/1710.09829

    arXivAutomatically verified via OpenAlex/arXiv – 2026-11-20

  2. Hinton, G. E., Sabour, S., & Frosst, N. (2018). Matrix capsules with EM routing. International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=HJWLfGWRb

    PublisherReference without a DOI or arXiv ID: checked manually against the publisher's data

  3. De Sousa Ribeiro, F., Duarte, K., Everett, M., Leontidis, G., & Shah, M. (2024). Object-centric learning with capsule networks: A survey. ACM Computing Surveys, 56(11), 1–291 (arXiv 2206.02664). https://doi.org/10.1145/3674500

    DOIarXivAutomatically verified via Crossref – 2026-11-20