Medical Architectures

U-Net → Attention U-Net → TransUNet → UNETR → Swin-UNETR → hybrid architectures.

Intermediate

Path goals

  • Understand the encoder-decoder structure and the skip connections.
  • Explain the attention gates in Attention U-Net.
  • Compare the convolutional encoder with the transformer encoder in segmentation.
  • Choose suitable evaluation metrics (Dice, IoU and HD95) while keeping patients separate between training and testing.

Prerequisites

  • The Foundations path.
  • The Computer Vision path (recommended).

Learning order

  1. U-Net U-Net

    A U-shaped segmentation architecture: a contracting path that captures context and an expanding path that restores spatial resolution, with skip conn…

  2. Attention U-Net Attention U-Net

    A U-Net with attention gates on the skip connections that learn to highlight the relevant regions and suppress the background before the features are…

  3. U-Net with Transformers U-Net + Transformer

    A family of segmentation models that keeps the U-Net shape (an encoder–decoder with skip connections) and introduces the transformer somewhere: the b…

  4. TransUNet TransUNet

    A medical segmentation model that uses a hybrid encoder: a CNN (ResNet-50) extracts a feature map, which is split into patches and passed through a V…

  5. UNETR UNETR

    A 3D segmentation model whose encoder is a pure ViT on 16³ volumetric patches; representations are taken from its layers (3, 6, 9, 12), reshaped and…

  6. Swin UNETR Swin UNETR

    A volumetric segmentation model whose encoder is a hierarchical Swin Transformer (shifted local windows and patch merging) that gives features at fiv…

Labs

Suggested projects

  1. Run the encyclopedia's U-Net, TransUNet and UNETR examples and compare Dice on the same synthetic data.
  2. Visit the Medical Image Analysis Lab to try models on real, openly licensed images.
  3. Design an evaluation protocol for brain tumour segmentation that splits the data by patient and reports Dice and HD95 with confidence intervals.

Path quiz

  1. True or false

    The skip connections in U-Net carry fine spatial detail from the encoder to the decoder.

  2. Mathematical

    Dice coefficient of two masks: the intersection is 30 pixels, the predicted mask 40 and the true mask 50. What is the Dice?

  3. Comparison

    What is the main difference between UNETR and TransUNet?

  4. True or false

    Mixing slices of the same patient between training and testing is an acceptable practice because it increases the data.