TL;DR
LeViT-UNet++ is a lightweight hybrid architecture that bridges the gap between convolutional and Transformer-based medical image segmentation. By integrating wavelet-based downsampling and an efficient depthwise-convolution decoder, it achieves competitive accuracy with a fraction of the parameters (only ~8M) and significantly higher inference speeds (up to 241 FPS).
What Problem Does This Paper Address?
Medical image segmentation is a high-stakes task where accuracy and speed are often at odds. Clinical applications like surgical navigation and real-time image-guided intervention require models that are both precise—capturing fine-grained anatomical boundaries—and efficient enough to run on standard hardware.
Current paradigms have distinct weaknesses:
- Convolutional Neural Networks (CNNs): Excellent at capturing local spatial details but struggle with long-range global context.
- Transformers: Superb at modeling global dependencies but are computationally expensive and often lose fine-grained details during the aggressive downsampling required to make them efficient.
The authors identify two specific bottlenecks in existing hybrid models like LeViT-UNet: conventional downsampling (like max pooling) inevitably discards high-frequency structural information, and standard decoders are often computationally redundant, using heavy convolutions to fuse features.
Methods
LeViT-UNet++ introduces two primary architectural innovations to solve these issues:
-
Haar Wavelet Multi-Scale Downsampling (HWMD): Instead of simple pooling, this module uses the Haar wavelet transform to decompose feature maps into low-frequency (coarse structures) and high-frequency (edges and fine details) components. By processing these components with parallel depthwise convolutions at multiple scales, the model preserves structural cues that are usually lost during spatial reduction.
Illustration of the HWMD module -
Depthwise-based Local and Global feature fusion and learning decoder (DLGD): The authors replaced standard, heavy convolutional blocks in the decoder with lightweight depthwise separable convolutions. This module progressively integrates global contextual representations from the Transformer blocks with local spatial features from the encoder's skip connections. This design reduces the decoder's parameter count by nearly 50% compared to the original LeViT-UNet.
The overall architecture follows a U-shaped encoder-decoder structure, using a hybrid encoder (CNN stem followed by Transformer blocks) to balance local and global feature extraction.
Datasets
The model was evaluated across four diverse benchmarks covering both 2D and 3D segmentation tasks:
- Synapse (Abdominal CT): Multi-organ segmentation of 8 abdominal organs.
- ACDC (Cardiac MRI): Segmentation of the left ventricle, myocardium, and right ventricle.
- LA2018 (Left Atrium MRI): A 3D segmentation challenge for the left atrium.
- MSD Heart (Cardiac MRI): Part of the Medical Segmentation Decathlon, focused on 3D heart structure.
Key Insights and Ideas
- Frequency Matters: The use of Haar wavelets suggests that "downsampling" shouldn't just be about reducing resolution; it should be about intelligent frequency decomposition. Keeping high-frequency information explicitly helps the model recover sharp boundaries later in the decoder.
- Hybrid Efficiency: The paper reinforces the idea that Transformers are most efficient when applied to low-resolution feature maps at the "bottleneck" of a network, while CNNs should handle the high-resolution "stems."
- Decoder Redundancy: A major "aha" moment is the realization that decoders in medical imaging models are often unnecessarily heavy. By switching to depthwise convolutions, the authors maintained (and even improved) accuracy while drastically cutting the computational budget.
Key Results
LeViT-UNet++ demonstrates a superior accuracy-efficiency trade-off compared to heavyweights like TransUNet and Swin-UNet:
- Synapse Dataset: Achieved a Dice Similarity Coefficient (DSC) of 76.57% with only 8.025M parameters and a blistering inference rate of 241.72 FPS. For comparison, TransUNet has ~67M parameters and runs at ~204 FPS with lower accuracy (75.31% DSC).
- ACDC Dataset: Attained a DSC of 88.95% and a 95th-percentile Hausdorff Distance (HD95) of 1.35, showing excellent boundary agreement.
- 3D Performance: On the LA2018 dataset, it boosted DSC from 81.50% (original LeViT-UNet) to 84.88% while reducing parameters by 75.6%.
Qualitatively, the model produces much crisper boundaries and fewer spurious fragments than its predecessors, as seen in the 3D reconstructions.
Limitations and Future Work
The authors acknowledge a few areas for improvement:
- Fusion Strategy: The current decoder relies on simple concatenation and depthwise convolution. More sophisticated, adaptive fusion mechanisms (like attention-guided selection) might further improve how local and global features interact.
- Dataset Diversity: While tested on four benchmarks, further validation on larger, more clinically diverse datasets is needed to prove generalizability across different pathologies and imaging hardware.
- Knowledge Distillation: The authors suggest that using knowledge distillation to transfer features from massive, pre-trained models into this lightweight architecture could be a promising next step for even higher performance.
Summary updated