Capsule Networks (CapsNet)
Networks that represent entities as capsules (vectors) whose length expresses the probability that the entity is present and whose direction expresses its properties (pose), and that link levels with a dynamic routing-by-agreement algorithm.
Overview
| Name | Capsule Networks |
|---|---|
| Year introduced | 2017 |
| Researchers | Sara Sabour, Nicholas Frosst, Geoffrey E. Hinton |
| Institution | Google Brain |
| Original paper | Dynamic routing between capsules arXiv |
| Data type | Images |
| Domains | Computer vision, Medical image analysis |
| Computational cost | Relatively high, because of the predictions for every pair of capsules and the routing iterations. |
| Data size | Small and medium datasets (MNIST, smallNORB). |
Problem it solves
Max pooling in a CNN loses information about the spatial relationships between parts, so the network may classify a face with scrambled parts as a face.
Background and evolution
What came before: Convolutional Neural Network
Core idea
Each capsule at the lower level (a part) predicts the pose of each higher capsule (a whole) through a transformation matrix. If the predictions of several parts agree on a similar pose, the coupling coefficients towards that capsule increase. The squash function keeps the vector length between 0 and 1.
Interactive diagram
A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.
Select a layer in the diagram.
Components
| Component | Role |
|---|---|
| Conv layer | Extracting low-level features |
| Primary Capsules | A convolution reshaped into small vectors |
| Transformation matrices | $W_{ij}$ for the part's prediction of the whole |
| Dynamic routing | Repeated updates of the coupling coefficients |
| Squash | Normalising the vector length |
| Margin loss | A loss on the lengths of the class capsules |
| Reconstruction decoder | Reconstructing the image as a regulariser |
Equations
1. The squash function
| $s_j$ | Total input of capsule j |
| $v_j$ | Its output |
2. Predictions and routing
| $c_{ij}$ | Coupling coefficient |
| $b_{ij}$ | Initial logit starting at zero |
3. Margin loss
| $T_k$ | 1 if class k is present |
| $m^+,m^-$ | 0.9 and 0.1 |
| $\lambda$ | 0.5 |
How it works, step by step
- An initial convolution extracts local features.
- The primary capsules are formed and squashed.
- Each capsule predicts the pose of the class capsules.
- Routing is repeated r times to update the coupling.
- The class is given by the longest capsule.
- A reconstruction decoder is added as a regulariser.
Strengths and limitations
Strengths
- An explicit representation of part–whole relationships and pose.
- Good results on MNIST and on overlapping digits (MultiMNIST) in the original paper.
Limitations
- Iterative routing is costly and hard to scale to large images.
- It has not shown a consistent advantage over modern CNNs on large image datasets.
When to use it, and when not?
Good for
- Studying spatial relationships and pose.
- Research experiments on small data.
Not suited for
- High-resolution images and huge datasets, where the cost is large.
Applications
| Domain | Use |
|---|---|
| Computer vision | Recognising overlapping digits and objects in different poses. |
| Medical image analysis | Experiments in classifying small medical images. |
PyTorch code
A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.
A miniature version for 16×16 images without the reconstruction decoder; the original paper uses MNIST with 28×28 images and 1152 primary capsules.
# Capsule Network with dynamic routing — a miniature version
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
# 2) Define the model
def squash(s, dim=-1):
n2 = (s ** 2).sum(dim, keepdim=True)
return n2 / (1 + n2) * s / torch.sqrt(n2 + 1e-8)
class CapsNet(nn.Module):
def __init__(self, n_classes=2, routing=3):
super().__init__()
self.conv = nn.Conv2d(1, 32, 5) # 16→12
self.primary = nn.Conv2d(32, 8 * 8, 5, stride=2) # 12→4 ⇒ 8 maps × 8D capsules
self.n_in, self.r = 8 * 4 * 4, routing
self.W = nn.Parameter(torch.randn(1, self.n_in, n_classes, 12, 8) * 0.05)
def forward(self, x):
h = F.relu(self.conv(x))
u = self.primary(h).view(x.shape[0], 8, 8, 4, 4).permute(0, 1, 3, 4, 2).reshape(x.shape[0], -1, 8)
u = squash(u) # B×128×8
u_hat = (self.W @ u[:, :, None, :, None]).squeeze(-1) # B×128×C×12
b = torch.zeros(u_hat.shape[:3])
for it in range(self.r): # routing by agreement
c = torch.softmax(b, dim=2)
v = squash((c.unsqueeze(-1) * u_hat).sum(1)) # B×C×12
if it < self.r - 1:
b = b + (u_hat * v.unsqueeze(1)).sum(-1)
return v.norm(dim=-1) # length of each capsule
def margin_loss(lengths, y, m_pos=0.9, m_neg=0.1, lam=0.5):
T = F.one_hot(y, lengths.shape[1]).float()
L = T * F.relu(m_pos - lengths) ** 2 + lam * (1 - T) * F.relu(lengths - m_neg) ** 2
return L.sum(1).mean()
# 3) Prepare the data
def make(n):
X = torch.randn(n, 1, 16, 16) * 0.3
y = torch.randint(0, 2, (n,))
pos = torch.randint(2, 14, (n,))
for i in range(n):
if y[i] == 0: X[i, 0, pos[i], 2:14] += 1.5
else: X[i, 0, 2:14, pos[i]] += 1.5
return X, y
X_train, y_train = make(384)
X_test, y_test = make(96)
model = CapsNet()
# 4) Margin loss and optimizer
opt = torch.optim.Adam(model.parameters(), lr=3e-3)
# 5) Training loop
for epoch in range(6):
model.train()
for i in range(0, len(X_train), 32):
xb, yb = X_train[i:i + 32], y_train[i:i + 32]
loss = margin_loss(model(xb), yb) # Forward Pass + Margin Loss
opt.zero_grad(); loss.backward(); opt.step()
print(f"epoch {epoch} loss={loss.item():.4f}")
# 6) Evaluation: class = longest capsule
model.eval()
with torch.no_grad():
acc = (model(X_test).argmax(1) == y_test).float().mean().item()
print(f"test accuracy = {acc:.3f}")
# 7) Inference: class-capsule lengths for one image
with torch.no_grad():
print("capsule lengths:", [round(v, 3) for v in model(X_test[:1])[0].tolist()])
This code was run on 2026-11-20 in 3.4 seconds (PyTorch (CPU), Python 3 – local build machine).
Sample output:
test accuracy = 1.000
Trainable parameters in the example: model = 76,672.
These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.
Try it and go deeper
Related architectures
Convolutional Neural Network
Convolutional Neural Network (CNN)
A network that uses small convolution filters with shared weights sliding over the image, so it captures local patterns with far fewer parameters than an MLP.
Hybrid Capsule Networks
Hybrid Capsule Networks (CNN-CapsNet)
Designs that use a strong convolutional backbone (such as ResNet) to extract features, then a capsule layer with dynamic routing on top to represent part–whole relations…
Attention Mechanism
Attention Mechanism
A mechanism that lets the model compute a weight for each input element according to its relevance to the current step, then take a weighted average, instead of compress…
No results match the filters.
Research ideas
- Evaluate the robustness of CapsNet to rotation compared with a CNN given equivalent data augmentation.
- Lightweight capsules for classifying small medical images, with a cost analysis.
Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.
Test your understanding
What does the length of a capsule vector represent in CapsNet?
Why? The squash function keeps the length between 0 and 1, and the direction represents the properties.
What is the squashed length of a vector of length 1?
Why? $1^2/(1+1^2)=0.5$.
What is the idea behind dynamic routing?
Why? $b_{ij}$ is updated with the dot product between the prediction and the output.
References
Sabour, S., Frosst, N., & Hinton, G. E. (2017). Dynamic routing between capsules. Advances in Neural Information Processing Systems (NeurIPS) 30. https://arxiv.org/abs/1710.09829
Hinton, G. E., Sabour, S., & Frosst, N. (2018). Matrix capsules with EM routing. International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=HJWLfGWRb
De Sousa Ribeiro, F., Duarte, K., Everett, M., Leontidis, G., & Shah, M. (2024). Object-centric learning with capsule networks: A survey. ACM Computing Surveys, 56(11), 1–291 (arXiv 2206.02664). https://doi.org/10.1145/3674500
