Medical Architectures
U-Net → Attention U-Net → TransUNet → UNETR → Swin-UNETR → hybrid architectures.
Path goals
- Understand the encoder-decoder structure and the skip connections.
- Explain the attention gates in Attention U-Net.
- Compare the convolutional encoder with the transformer encoder in segmentation.
- Choose suitable evaluation metrics (Dice, IoU and HD95) while keeping patients separate between training and testing.
Prerequisites
- The Foundations path.
- The Computer Vision path (recommended).
Learning order
- U-Net U-Net
A U-shaped segmentation architecture: a contracting path that captures context and an expanding path that restores spatial resolution, with skip conn…
- Attention U-Net Attention U-Net
A U-Net with attention gates on the skip connections that learn to highlight the relevant regions and suppress the background before the features are…
- U-Net with Transformers U-Net + Transformer
A family of segmentation models that keeps the U-Net shape (an encoder–decoder with skip connections) and introduces the transformer somewhere: the b…
- TransUNet TransUNet
A medical segmentation model that uses a hybrid encoder: a CNN (ResNet-50) extracts a feature map, which is split into patches and passed through a V…
- UNETR UNETR
A 3D segmentation model whose encoder is a pure ViT on 16³ volumetric patches; representations are taken from its layers (3, 6, 9, 12), reshaped and…
- Swin UNETR Swin UNETR
A volumetric segmentation model whose encoder is a hierarchical Swin Transformer (shifted local windows and patch merging) that gives features at fiv…
Labs
Suggested projects
- Run the encyclopedia's U-Net, TransUNet and UNETR examples and compare Dice on the same synthetic data.
- Visit the Medical Image Analysis Lab to try models on real, openly licensed images.
- Design an evaluation protocol for brain tumour segmentation that splits the data by patient and reports Dice and HD95 with confidence intervals.
Path quiz
The skip connections in U-Net carry fine spatial detail from the encoder to the decoder.
Why? They restore the detail lost during downsampling, which is what improves the segmentation boundaries.
Dice coefficient of two masks: the intersection is 30 pixels, the predicted mask 40 and the true mask 50. What is the Dice?
Why? $2\times30/(40+50)=60/90\approx0.667$.
What is the main difference between UNETR and TransUNet?
Why? TransUNet is hybrid: a CNN then a Transformer, while UNETR passes the image patches directly to a Transformer.
Mixing slices of the same patient between training and testing is an acceptable practice because it increases the data.
Why? This is data leakage that inflates the results; the split must be done at the patient level.
