Hybrid Capsule Networks (CNN-CapsNet)
Designs that use a strong convolutional backbone (such as ResNet) to extract features, then a capsule layer with dynamic routing on top to represent part–whole relationships.
Overview
| Name | Hybrid Capsule Networks |
|---|---|
| Year introduced | 2017 |
| Researchers | A hybrid design built on CapsNet (Sabour et al., 2017) |
| Original paper | Dynamic routing between capsules arXiv |
| Data type | Images |
| Domains | Computer vision, Medical image analysis |
Problem it solves
The original CapsNet relies on a single shallow convolution, so it is weak on complex images; deep CNNs lose pose relationships at pooling.
Background and evolution
What came before: Capsule Networks, Residual Network
Core idea
"Hybrid" here describes a family of designs, not a single paper: a pretrained CNN backbone extracts rich feature maps, which are reshaped into primary capsules, and then a layer of class capsules with routing is applied. The aim is to combine the strength of CNNs in local features with the ability of capsules to represent pose.
Interactive diagram
A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.
Select a layer in the diagram.
Components
| Component | Role |
|---|---|
| CNN backbone | Pretrained ResNet or VGG |
| Primary capsules | Reshaping the feature maps |
| Class capsules | Dynamic routing |
| Margin loss | As in CapsNet |
Equations
1. Capsules from the feature maps
| $f_{\mathrm{CNN}}$ | The backbone |
| $u_i$ | Primary capsule i |
How it works, step by step
- The image passes through the CNN backbone.
- The features are turned into primary capsules.
- Routing is run towards the class capsules.
- The margin loss is computed.
Strengths and limitations
Strengths
- Stronger features than the original CapsNet.
- Pretrained weights can be used.
Limitations
- The routing cost remains.
- The benefit over a CNN alone is not guaranteed and needs fairly tuned experiments.
When to use it, and when not?
Good for
- Research experiments on small or medium medical images.
Not suited for
- When there are no resources for a fair comparison with a strong CNN.
Applications
| Domain | Use |
|---|---|
| Medical image analysis | Medical image classification with pose representation. |
| Computer vision | Recognising objects in varying poses. |
PyTorch code
A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.
# Hybrid CNN-CapsNet: a miniature ResNet backbone + capsules
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
# 2) Define the model
def squash(s, dim=-1):
n2 = (s ** 2).sum(dim, keepdim=True)
return n2 / (1 + n2) * s / torch.sqrt(n2 + 1e-8)
class Backbone(nn.Module): # a small convolutional backbone with a residual connection
def __init__(self):
super().__init__()
self.c1 = nn.Sequential(nn.Conv2d(1, 32, 3, padding=1), nn.BatchNorm2d(32), nn.ReLU())
self.c2 = nn.Sequential(nn.Conv2d(32, 32, 3, padding=1), nn.BatchNorm2d(32))
def forward(self, x):
h = self.c1(x)
return F.max_pool2d(F.relu(h + self.c2(h)), 2) # 32×8×8
class HybridCaps(nn.Module):
def __init__(self, n_classes=2, r=3):
super().__init__()
self.backbone = Backbone()
self.primary = nn.Conv2d(32, 8 * 8, 3, stride=2, padding=1) # 8 maps × 8D on 4×4
self.W = nn.Parameter(torch.randn(1, 128, n_classes, 12, 8) * 0.05)
self.r = r
def forward(self, x):
u = self.primary(self.backbone(x)).view(x.shape[0], 8, 8, 16).permute(0, 1, 3, 2).reshape(x.shape[0], 128, 8)
u_hat = (self.W @ squash(u)[:, :, None, :, None]).squeeze(-1)
b = torch.zeros(u_hat.shape[:3])
for it in range(self.r):
v = squash((torch.softmax(b, 2).unsqueeze(-1) * u_hat).sum(1))
if it < self.r - 1: b = b + (u_hat * v.unsqueeze(1)).sum(-1)
return v.norm(dim=-1)
def margin_loss(lengths, y):
T = F.one_hot(y, lengths.shape[1]).float()
return (T * F.relu(0.9 - lengths) ** 2 + 0.5 * (1 - T) * F.relu(lengths - 0.1) ** 2).sum(1).mean()
# 3) Prepare the data
def make(n):
X = torch.randn(n, 1, 16, 16) * 0.3
y = torch.randint(0, 2, (n,))
pos = torch.randint(2, 14, (n,))
for i in range(n):
if y[i] == 0: X[i, 0, pos[i], 2:14] += 1.5
else: X[i, 0, 2:14, pos[i]] += 1.5
return X, y
X_train, y_train = make(384)
X_test, y_test = make(96)
model = HybridCaps()
# 4) Optimizer (the loss: margin_loss)
opt = torch.optim.Adam(model.parameters(), lr=3e-3)
# 5) Training loop
for epoch in range(5):
model.train()
for i in range(0, len(X_train), 32):
loss = margin_loss(model(X_train[i:i + 32]), y_train[i:i + 32])
opt.zero_grad(); loss.backward(); opt.step()
print(f"epoch {epoch} loss={loss.item():.4f}")
# 6) Evaluation
model.eval()
with torch.no_grad():
acc = (model(X_test).argmax(1) == y_test).float().mean().item()
print(f"test accuracy = {acc:.3f}")
# 7) Inference
with torch.no_grad():
print("predicted class:", model(X_test[:1]).argmax(1).item())
This code was run on 2026-11-20 in 3.6 seconds (PyTorch (CPU), Python 3 – local build machine).
Sample output:
test accuracy = 1.000
Trainable parameters in the example: model = 52,768.
These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.
Try it and go deeper
Related architectures
Capsule Networks
Capsule Networks (CapsNet)
Networks that represent entities as capsules (vectors) whose length expresses the probability that the entity is present and whose direction expresses its properties (po…
Residual Network
Residual Network (ResNet)
It adds skip connections that make each block learn the residual difference F(x) instead of the full transformation, so training networks with hundreds of layers becomes…
Convolutional Neural Network
Convolutional Neural Network (CNN)
A network that uses small convolution filters with shared weights sliding over the image, so it captures local patterns with far fewer parameters than an MLP.
No results match the filters.
Research ideas
- A fair comparison between ResNet and ResNet+Capsules on histopathology data with the same compute budget.
Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.
Test your understanding
What is the role of the CNN backbone in the hybrid capsule network?
Why? The capsules are built on top of the feature maps.
Is "Hybrid Capsule Network" the name of one specific research paper?
Why? There are many hybrid designs with different names.
What challenge remains in hybrid designs?
Why? Routing requires predictions for every pair of capsules.
References
Sabour, S., Frosst, N., & Hinton, G. E. (2017). Dynamic routing between capsules. Advances in Neural Information Processing Systems (NeurIPS) 30. https://arxiv.org/abs/1710.09829
De Sousa Ribeiro, F., Duarte, K., Everett, M., Leontidis, G., & Shah, M. (2024). Object-centric learning with capsule networks: A survey. ACM Computing Surveys, 56(11), 1–291 (arXiv 2206.02664). https://doi.org/10.1145/3674500
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778. https://doi.org/10.1109/CVPR.2016.90
