Simple is what you need for efficient and accurate medical image segmentation

Xiang Yu, Yayan Chen, Guannan He, Qing Zeng, Yue Qin, Meiling Liang, Dandan Luo, Yimei Liao, Zeyu Ren, Kang Cheng, Delong Yang, Bocheng Liang, Bin Pu, Shengli Li, Ying Yuan

TL;DR

SimpleUNet is an ultra-lightweight medical image segmentation model that achieves state-of-the-art accuracy with a fraction of the parameters used by traditional models. By replacing U-Net’s exponential width growth with a fixed-width design and pruning redundant information in skip connections, the authors produced a variant as small as 16 KB that outperforms much larger benchmarks.

What Problem Does This Paper Address?

In the world of medical AI, the trend has been "bigger is better." Researchers often chase higher accuracy by adding complex attention modules, Transformers, or state-space models (like Mamba). While these models perform well, they are often too "heavy" for practical clinical use, especially on resource-constrained hardware like mobile ultrasound machines or edge devices.

The authors identify a fundamental inefficiency in the classic U-Net architecture: as the network goes deeper, the number of channels doubles at every stage. This leads to exponential parameter growth. Furthermore, the standard practice of concatenating all feature maps through skip connections introduces significant information redundancy, forcing the decoder to work harder (and use more parameters) to process unnecessary data.

Methods

SimpleUNet strips back the complexity of modern models and introduces three elegant architectural changes:

  1. Fixed-Width Architecture: Unlike the standard U-Net, which doubles its channel count at each downsampling step, SimpleUNet maintains a constant width throughout the encoder. This shifts the parameter growth from exponential to linear.
  2. Efficient Feature Selection: Instead of passing every feature map through skip connections, the model uses $1 \times 1$ convolutions to select only the most representative features (a "partial" selection). This drastically reduces the number of parameters required in the decoding stage.
  3. Adaptive Feature Fusion (AFF): To ensure the model doesn't lose accuracy while shrinking, the authors introduced learnable channel-wise weights ($\alpha$ and $\beta$). This acts as a "cost-minimal" attention mechanism, allowing the network to dynamically prioritize the most important information during the fusion of shortcut and deep features.

Datasets

The model was rigorously evaluated across seven public datasets covering three distinct clinical areas:

  • Merged Breast Dataset (MBD): A curated collection of ~3,000 ultrasound images from BUSI, DatasetB, BrEaST, and BUS-BRA.
  • Skin Lesion Datasets: ISIC 2017 and ISIC 2018, which feature dermoscopic images.
  • Endoscopic Polyp Segmentation: The KVASIR-SEG dataset, containing gastrointestinal polyp images.

Key Insights and Ideas

  • Redundancy is a Choice, Not a Necessity: The most surprising finding is that concatenating only partial features (e.g., a 50% selection rate) often leads to better segmentation performance than using the full set. This suggests that standard U-Nets are hindered by "noise" in their skip connections.
  • Simplicity as a Design Philosophy: The authors argue that "obscure modules" aren't always necessary. By refining the basic structural logic of U-Net, they achieved better results than models using complex Transformers.
  • Extreme Scalability: The architecture is highly "interpretable" and scalable. Researchers can tune the model width and selection rate to fit anything from a tiny 16 KB footprint to a more robust 0.67 MB version depending on the deployment environment.

Key Results

  • Record-Breaking Compactness: The 16 KB variant of SimpleUNet outperformed LBUNet (a popular lightweight benchmark) across multiple datasets.
  • David vs. Goliath: The 0.67 MB variant achieved a mean Dice Similarity Coefficient (mDSC) of 85.76% on breast lesion datasets. This surpassed TransUNet, a heavy Transformer-based model that is roughly 140 times larger (93.23 MB).
  • Efficiency: On the KVASIR-SEG dataset, SimpleUNet reached an mDice of 86.46%, dominating other state-of-the-art models while maintaining high computational efficiency (8.60 GFLOPs for the 0.67 MB version).
  • Qualitative Superiority: Visual results show that SimpleUNet is particularly good at avoiding the "over-segmentation" or "under-segmentation" errors common in other lightweight models when dealing with varied lesion sizes.

Limitations and Future Work

While SimpleUNet is highly efficient, the authors note a few areas for further optimization:

  • Initial Stage Efficiency: The first stage of the convolution does not currently use spatial reduction or grouped depthwise convolutions, which could further lower the computational floor.
  • Advanced Fusion: While the Adaptive Feature Fusion (AFF) is effective, the authors suggest exploring spatial or gated attention mechanisms to see if they can provide even better accuracy without ballooning the parameter count.
  • Deep Supervision: The current training does not utilize deep supervision (calculating loss at multiple stages), which is a common trick in other lightweight models to speed up convergence and improve performance.

The code is publicly available at: https://github.com/Frankyu5666666/SimpleUNet

Summary updated

Abstract

While modern segmentation models often prioritize performance over practicality, we advocate a design philosophy prioritizing simplicity and efficiency, and attempted high performance segmentation model design. This paper presents SimpleUNet, a scalable ultra-lightweight medical image segmentation model with three key innovations: (1) A partial feature selection mechanism in skip connections for redundancy reduction while enhancing segmentation performance; (2) A fixed-width architecture that prevents exponential parameter growth across network stages; (3) An adaptive feature fusion module achieving enhanced representation with minimal computational overhead. With a record-breaking 16 KB parameter configuration, SimpleUNet outperforms LBUNet and other lightweight benchmarks across multiple public datasets. The 0.67 MB variant achieves superior efficiency (8.60 GFLOPs) and accuracy, attaining a mean DSC/IoU of 85.76%/75.60% on multi-center breast lesion datasets, surpassing both U-Net and TransUNet. Evaluations on skin lesion datasets (ISIC 2017/2018: mDice 84.86%/88.77%) and endoscopic polyp segmentation (KVASIR-SEG: 86.46%/76.48% mDice/mIoU) confirm consistent dominance over state-of-the-art models. This work demonstrates that extreme model compression need not compromise performance, providing new insights for efficient and accurate medical image segmentation. Codes can be found at https://github.com/Frankyu5666666/SimpleUNet.
BibTeX
@inproceedings{yu2025simple,
  author = {Xiang Yu and Yayan Chen and Guannan He and Qing Zeng and Yue Qin and Meiling Liang and Dandan Luo and Yimei Liao and Zeyu Ren and Kang Cheng and Delong Yang and Bocheng Liang and Bin Pu and Shengli Li and Ying Yuan},
  title = {{Simple is what you need for efficient and accurate medical image segmentation}},
  booktitle = {Expert},
  year = {2025},
  doi = {10.1016/j.eswa.2025.130687},
}

Loading related content...

Figure