Attention U-Net
A U-Net with attention gates on the skip connections that learn to highlight the relevant regions and suppress the background before the features are merged.
Overview
| Name | Attention U-Net |
|---|---|
| Year introduced | 2018 |
| Researchers | Ozan Oktay, Jo Schlemper, Loic Le Folgoc et al. |
| Original paper | Attention U-Net: Learning where to look for the pancreas arXiv |
| Data type | Images |
| Domains | Medical image analysis |
Problem it solves
Small organs such as the pancreas occupy a small part of the image, so the skip connections carry many unhelpful background features.
Background and evolution
What came before: U-Net
- 2018
It was introduced at MIDL 2018 and tested on pancreas segmentation in CT images. [ref]
Core idea
Before the encoder map $x$ is concatenated with the decoder, a weight map $\alpha\in[0,1]$ is computed for each pixel using a gating signal $g$ from the deeper level. $x$ is multiplied by $\alpha$ so that only the important regions pass.
Interactive diagram
A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.
Select a layer in the diagram.
Components
| Component | Role |
|---|---|
| Encoder/Decoder | As in U-Net |
| Gating Signal g | Features of the deeper level |
| Attention Gate | Additive fusion, then Sigmoid |
| Skip Connection | Concatenating the features after multiplying by the weights |
Equations
1. Attention gate
| $x_i$ | Encoder features at pixel i |
| $g_i$ | Gating signal |
| $\sigma_1$ | ReLU |
| $\sigma_2$ | Sigmoid |
How it works, step by step
- The input passes through the U-Net encoder.
- At each decoder level the gating signal is computed.
- The α map is computed from x and g.
- The skip features are multiplied by α.
- They are concatenated with the decoder and upsampling continues.
Strengths and limitations
Strengths
- Improves the segmentation of small targets.
- A light addition to U-Net.
- Gives attention maps that can be displayed.
Limitations
- Increases the cost slightly.
- The benefit depends on the task and is not guaranteed.
When to use it, and when not?
Good for
- Segmenting small organs or lesions in a large background.
Not suited for
- When the targets occupy most of the image.
Applications
| Domain | Use |
|---|---|
| Medical image analysis | Segmenting the pancreas and small lesions in CT and MRI. |
PyTorch code
A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.
# A miniature Attention U-Net
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
# 2) Define the model
def block(ci, co):
return nn.Sequential(nn.Conv2d(ci, co, 3, padding=1), nn.BatchNorm2d(co), nn.ReLU(),
nn.Conv2d(co, co, 3, padding=1), nn.BatchNorm2d(co), nn.ReLU())
class AttentionGate(nn.Module):
def __init__(self, cx, cg, ci):
super().__init__()
self.wx, self.wg = nn.Conv2d(cx, ci, 1), nn.Conv2d(cg, ci, 1)
self.psi = nn.Conv2d(ci, 1, 1)
def forward(self, x, g):
a = torch.sigmoid(self.psi(F.relu(self.wx(x) + self.wg(g)))) # B×1×H×W
return x * a
class AttUNet(nn.Module):
def __init__(self, c=1, k=2):
super().__init__()
self.e1, self.e2, self.mid = block(c, 8), block(8, 16), block(16, 32)
self.up2, self.ag2, self.d2 = nn.ConvTranspose2d(32, 16, 2, 2), AttentionGate(16, 16, 8), block(32, 16)
self.up1, self.ag1, self.d1 = nn.ConvTranspose2d(16, 8, 2, 2), AttentionGate(8, 8, 4), block(16, 8)
self.out = nn.Conv2d(8, k, 1)
def forward(self, x):
e1 = self.e1(x); e2 = self.e2(F.max_pool2d(e1, 2)); m = self.mid(F.max_pool2d(e2, 2))
u2 = self.up2(m); d2 = self.d2(torch.cat([u2, self.ag2(e2, u2)], 1))
u1 = self.up1(d2); d1 = self.d1(torch.cat([u1, self.ag1(e1, u1)], 1))
return self.out(d1)
# 3) Prepare the data
# Small targets (circles with a radius of 2–3) in a background with distracting squares
def make(n, S=32):
yy, xx = torch.meshgrid(torch.arange(S), torch.arange(S), indexing='ij')
cx, cy, r = torch.randint(6, 26, (n,)), torch.randint(6, 26, (n,)), torch.randint(2, 4, (n,))
mask = ((xx[None] - cx[:, None, None]) ** 2 + (yy[None] - cy[:, None, None]) ** 2 <= r[:, None, None] ** 2).long()
X = mask[:, None].float() * 1.5 + 0.4 * torch.randn(n, 1, S, S)
for i in range(n):
a, b = torch.randint(0, S - 6, (2,)).tolist()
X[i, 0, a:a + 6, b:b + 6] += 0.6 # distractor: a fainter square
return X, mask
X_train, y_train = make(256)
X_test, y_test = make(64)
model = AttUNet()
# 4) The loss: weighted CE because the target is small
loss_fn = nn.CrossEntropyLoss(weight=torch.tensor([1.0, 5.0]))
opt = torch.optim.Adam(model.parameters(), lr=1e-2)
# 5) Training loop
for epoch in range(8):
model.train()
for i in range(0, len(X_train), 32):
xb, yb = X_train[i:i + 32], y_train[i:i + 32]
loss = loss_fn(model(xb), yb)
opt.zero_grad(); loss.backward(); opt.step()
print(f"epoch {epoch} loss={loss.item():.4f}")
# 6) Evaluation with the Dice coefficient
model.eval()
with torch.no_grad():
pred = model(X_test).argmax(1)
inter = ((pred == 1) & (y_test == 1)).sum().item()
print(f"test Dice = {2 * inter / max(1, (pred == 1).sum().item() + (y_test == 1).sum().item()):.3f}")
# 7) Inference
with torch.no_grad():
print("foreground pixels in first test image:", int(model(X_test[:1]).argmax(1).sum()))
This code was run on 2026-11-20 in 5.5 seconds (PyTorch (CPU), Python 3 – local build machine).
Sample output:
test Dice = 0.974
Trainable parameters in the example: model = 30,008.
These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.
Try it and go deeper
- Medical Image Analysis Lab Another section of the site
Related architectures
U-Net
U-Net
A U-shaped segmentation architecture: a contracting path that captures context and an expanding path that restores spatial resolution, with skip connections that carry f…
TransUNet
TransUNet
A medical segmentation model that uses a hybrid encoder: a CNN (ResNet-50) extracts a feature map, which is split into patches and passed through a ViT, followed by a ca…
Attention-Augmented Convolutional Networks
Attention-based CNN (Attention CNN)
Convolutional networks that add lightweight attention modules that reweight the channels (which features matter most) or the positions (where the model looks), at a very…
No results match the filters.
Research ideas
- Evaluate attention gates against Swin-UNETR for segmenting small lesions, with a sensitivity analysis by lesion size.
Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.
Test your understanding
Where are the attention gates placed in Attention U-Net?
Why? They filter the encoder features before they are merged.
What is the range of the attention coefficients α in the gate?
Why? The last function in the gate is Sigmoid.
For which kind of task do attention gates help most?
Why? This is the original motivation in the paper (the pancreas in CT).
References
Oktay, O., Schlemper, J., Le Folgoc, L., Lee, M., Heinrich, M., Misawa, K., et al. (2018). Attention U-Net: Learning where to look for the pancreas. Medical Imaging with Deep Learning (MIDL). https://arxiv.org/abs/1804.03999
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention (MICCAI), LNCS 9351, 234–241. https://doi.org/10.1007/978-3-319-24574-4_28
