IntermediateHybrid Architectures

Hybrid Capsule Networks (CNN-CapsNet)

Designs that use a strong convolutional backbone (such as ResNet) to extract features, then a capsule layer with dynamic routing on top to represent part–whole relationships.

Overview

NameHybrid Capsule Networks
Year introduced2017
ResearchersA hybrid design built on CapsNet (Sabour et al., 2017)
Original paperDynamic routing between capsules arXiv
Data typeImages
DomainsComputer vision, Medical image analysis

Problem it solves

The original CapsNet relies on a single shallow convolution, so it is weak on complex images; deep CNNs lose pose relationships at pooling.

Background and evolution

What came before: Capsule Networks, Residual Network

  1. 2017

    The dynamic routing on which the top layer is built. [ref]

  2. 2024

    The capsule survey reviews a number of hybrid designs and their limits. [ref]

Core idea

"Hybrid" here describes a family of designs, not a single paper: a pretrained CNN backbone extracts rich feature maps, which are reshaped into primary capsules, and then a layer of class capsules with routing is applied. The aim is to combine the strength of CNNs in local features with the ability of capsules to represent pose.

Example: In histopathology image classification, ResNet extracts cell patterns, and the higher capsules combine these patterns with their relative positions.

Interactive diagram

A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.

Select a layer in the diagram.

Components

ComponentRole
CNN backbonePretrained ResNet or VGG
Primary capsulesReshaping the feature maps
Class capsulesDynamic routing
Margin lossAs in CapsNet

Equations

1. Capsules from the feature maps

$$u_i=\mathrm{squash}\!\left(\mathrm{reshape}(\mathrm{Conv}(f_{\mathrm{CNN}}(x)))_i\right)$$

Symbols
$f_{\mathrm{CNN}}$The backbone
$u_i$Primary capsule i
Dimensions
Each $u_i$ is a vector of fixed length
Numerical example
A $64\times4\times4$ map is split into $8\times4\times4=128$ capsules of length 8.
Relation to the architecture
The meeting point of the two parts.

How it works, step by step

  1. The image passes through the CNN backbone.
  2. The features are turned into primary capsules.
  3. Routing is run towards the class capsules.
  4. The margin loss is computed.

Strengths and limitations

Strengths

  • Stronger features than the original CapsNet.
  • Pretrained weights can be used.

Limitations

  • The routing cost remains.
  • The benefit over a CNN alone is not guaranteed and needs fairly tuned experiments.

When to use it, and when not?

Good for

  • Research experiments on small or medium medical images.

Not suited for

  • When there are no resources for a fair comparison with a strong CNN.

Applications

DomainUse
Medical image analysisMedical image classification with pose representation.
Computer visionRecognising objects in varying poses.

PyTorch code

A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.

Python (PyTorch)
# Hybrid CNN-CapsNet: a miniature ResNet backbone + capsules
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F

torch.manual_seed(0)

# 2) Define the model
def squash(s, dim=-1):
    n2 = (s ** 2).sum(dim, keepdim=True)
    return n2 / (1 + n2) * s / torch.sqrt(n2 + 1e-8)

class Backbone(nn.Module):                          # a small convolutional backbone with a residual connection
    def __init__(self):
        super().__init__()
        self.c1 = nn.Sequential(nn.Conv2d(1, 32, 3, padding=1), nn.BatchNorm2d(32), nn.ReLU())
        self.c2 = nn.Sequential(nn.Conv2d(32, 32, 3, padding=1), nn.BatchNorm2d(32))
    def forward(self, x):
        h = self.c1(x)
        return F.max_pool2d(F.relu(h + self.c2(h)), 2)   # 32×8×8

class HybridCaps(nn.Module):
    def __init__(self, n_classes=2, r=3):
        super().__init__()
        self.backbone = Backbone()
        self.primary = nn.Conv2d(32, 8 * 8, 3, stride=2, padding=1)        # 8 maps × 8D on 4×4
        self.W = nn.Parameter(torch.randn(1, 128, n_classes, 12, 8) * 0.05)
        self.r = r
    def forward(self, x):
        u = self.primary(self.backbone(x)).view(x.shape[0], 8, 8, 16).permute(0, 1, 3, 2).reshape(x.shape[0], 128, 8)
        u_hat = (self.W @ squash(u)[:, :, None, :, None]).squeeze(-1)
        b = torch.zeros(u_hat.shape[:3])
        for it in range(self.r):
            v = squash((torch.softmax(b, 2).unsqueeze(-1) * u_hat).sum(1))
            if it < self.r - 1: b = b + (u_hat * v.unsqueeze(1)).sum(-1)
        return v.norm(dim=-1)

def margin_loss(lengths, y):
    T = F.one_hot(y, lengths.shape[1]).float()
    return (T * F.relu(0.9 - lengths) ** 2 + 0.5 * (1 - T) * F.relu(lengths - 0.1) ** 2).sum(1).mean()

# 3) Prepare the data
def make(n):
    X = torch.randn(n, 1, 16, 16) * 0.3
    y = torch.randint(0, 2, (n,))
    pos = torch.randint(2, 14, (n,))
    for i in range(n):
        if y[i] == 0: X[i, 0, pos[i], 2:14] += 1.5
        else:         X[i, 0, 2:14, pos[i]] += 1.5
    return X, y
X_train, y_train = make(384)
X_test, y_test = make(96)
model = HybridCaps()

# 4) Optimizer (the loss: margin_loss)
opt = torch.optim.Adam(model.parameters(), lr=3e-3)

# 5) Training loop
for epoch in range(5):
    model.train()
    for i in range(0, len(X_train), 32):
        loss = margin_loss(model(X_train[i:i + 32]), y_train[i:i + 32])
        opt.zero_grad(); loss.backward(); opt.step()
    print(f"epoch {epoch}  loss={loss.item():.4f}")

# 6) Evaluation
model.eval()
with torch.no_grad():
    acc = (model(X_test).argmax(1) == y_test).float().mean().item()
print(f"test accuracy = {acc:.3f}")

# 7) Inference
with torch.no_grad():
    print("predicted class:", model(X_test[:1]).argmax(1).item())

This code was run on 2026-11-20 in 3.6 seconds (PyTorch (CPU), Python 3 – local build machine).

Sample output:

test accuracy = 1.000

Trainable parameters in the example: model = 52,768.

These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.

Try it and go deeper

Related architectures

Intermediate2017

Capsule Networks

Capsule Networks (CapsNet)

Networks that represent entities as capsules (vectors) whose length expresses the probability that the entity is present and whose direction expresses its properties (po…

Advanced Neural Architectures

Basic2016

Residual Network

Residual Network (ResNet)

It adds skip connections that make each block learn the residual difference F(x) instead of the full transformation, so training networks with hundreds of layers becomes…

Foundations

Beginner1998

Convolutional Neural Network

Convolutional Neural Network (CNN)

A network that uses small convolution filters with shared weights sliding over the image, so it captures local patterns with far fewer parameters than an MLP.

Foundations

Research ideas

  • A fair comparison between ResNet and ResNet+Capsules on histopathology data with the same compute budget.

Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.

Test your understanding

  1. Concept

    What is the role of the CNN backbone in the hybrid capsule network?

  2. Concept

    Is "Hybrid Capsule Network" the name of one specific research paper?

  3. Concept

    What challenge remains in hybrid designs?

References

  1. Sabour, S., Frosst, N., & Hinton, G. E. (2017). Dynamic routing between capsules. Advances in Neural Information Processing Systems (NeurIPS) 30. https://arxiv.org/abs/1710.09829

    arXivAutomatically verified via OpenAlex/arXiv – 2026-11-20

  2. De Sousa Ribeiro, F., Duarte, K., Everett, M., Leontidis, G., & Shah, M. (2024). Object-centric learning with capsule networks: A survey. ACM Computing Surveys, 56(11), 1–291 (arXiv 2206.02664). https://doi.org/10.1145/3674500

    DOIarXivAutomatically verified via Crossref – 2026-11-20

  3. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778. https://doi.org/10.1109/CVPR.2016.90

    DOIarXivAutomatically verified via Crossref – 2026-11-20