Focus U-Net: A novel dual attention-gated CNN for polyp segmentation\n during colonoscopy

Michael Yeung, Evis Sala, Carola‐Bibiane Schönlieb, Leonardo Rundo

Colorectal cancer remains a leading cause of cancer-related deaths worldwide, yet it is highly treatable if caught early. The gold standard for screening is the colonoscopy, but there’s a catch: human error. Clinicians miss polyps at a rate of up to 26%, particularly when they are small or flat.

In a new paper, researchers from the University of Cambridge introduce Focus U-Net, a deep learning architecture designed to act as a high-precision "second pair of eyes" for endoscopists, significantly outperforming previous state-of-the-art models in identifying and delineating polyps.

TL;DR

Focus U-Net introduces a novel "Focus Gate" that combines spatial and channel-based attention with a tunable focal parameter to suppress background noise. By pairing this with a specialized "Hybrid Focal loss," the model achieves a 14% improvement in segmentation accuracy over previous benchmarks when tested across diverse, real-world datasets.

What Problem Does This Paper Address?

The primary challenge in automated polyp segmentation is class imbalance. In a typical colonoscopy frame, the polyp (the "foreground") is often tiny compared to the surrounding healthy tissue (the "background"). Standard AI models often get "distracted" by the background or fail to capture the high diversity in polyp shapes, textures, and colors.

Furthermore, many high-performing models are too computationally heavy for real-time use. To be useful in a clinical setting, an AI needs to be both incredibly accurate and fast enough to process live video without lag.

Methods

The authors built upon the classic U-Net architecture—the workhorse of medical image segmentation—and introduced several key innovations:

  • The Focus Gate (FG): This is the paper's centerpiece. It processes "where" (spatial attention) and "what" (channel attention) in parallel, mimicking the dual pathways of the human visual system.
  • The Focal Parameter ($\gamma$): Borrowing a concept from loss functions, the authors applied a "focal filter" directly to the attention weights. This allows the model to aggressively down-weight irrelevant background regions while preserving focus on the boundaries of the polyp.
  • Hybrid Focal Loss (HFL): To handle the fact that polyps are small and rare, the authors created a compound loss function that combines Focal Loss and Focal Tversky Loss. This forces the model to care more about hard-to-segment pixels and reduces the penalty for "easy" background pixels.
  • Structural Tweaks: They added short-range skip connections and "deep supervision" (calculating loss at multiple depths of the network) to prevent the "vanishing gradient" problem, ensuring the model learns effectively even in its deepest layers.

Datasets

The researchers didn't just test on one "clean" dataset; they used five diverse public benchmarks to ensure their model could generalize to different camera types and patient populations:

  1. CVC-ClinicDB (612 frames)
  2. Kvasir-SEG (1,000 images)
  3. CVC-ColonDB (300 images)
  4. ETIS-Larib PolypDB (300 images)
  5. EndoScene / CVC-T (182 frames)

Crucially, they also evaluated the model on a combined dataset of all five, which is a much harder test of a model's ability to handle "unseen" data.

Key Insights and Ideas

  • Parallel Attention is Better: Most models use attention serially (one after the other). Focus U-Net shows that processing spatial and channel information in parallel provides a more robust "recalibration" of features.
  • Attention Needs a "Focus": Simply having an attention gate isn't enough. By adding the tunable focal parameter, the researchers found they could manually control how "aggressive" the model is at ignoring the background.
  • Generalization is the Real Test: Many models perform well on a single dataset but fail in the real world. Focus U-Net’s massive jump in performance on the combined dataset suggests it is much better at handling the "messiness" of different clinical environments.

Key Results

The Focus U-Net set new records across the board:

  • SOTA on CVC-ClinicDB: Achieved a mean Dice Similarity Coefficient (mDSC) of 0.941.
  • SOTA on Kvasir-SEG: Achieved an mDSC of 0.910.
  • The "Combined" Breakthrough: On the mixture of all five datasets, Focus U-Net achieved an mDSC of 0.878—a 14% improvement over the previous leader (HarDNet-MSEG).
  • Tough Datasets: On the notoriously difficult ETIS-Larib dataset, the model showed a staggering 23% improvement over previous state-of-the-art results.

Limitations and Future Work

While the results are impressive, the authors note a few hurdles before this hits the operating room:

  • The "Empty Frame" Problem: The datasets used for training all contained at least one polyp. In a real colonoscopy, most of the video is just healthy tissue. Training on "negative" frames is necessary to reduce potential false positives.
  • Live Video Integration: The study focused on static images. The next step is validating the model on live, high-frame-rate video streams to ensure the "lightweight" design holds up under real-time pressure.
  • Beyond the Colon: The authors suggest the Focus U-Net architecture is general enough to be adapted for other medical tasks, such as segmenting lesions in CT or MRI scans where class imbalance is also a major issue.

Summary updated

Abstract

Background: Colonoscopy remains the gold-standard screening for colorectal\ncancer. However, significant miss rates for polyps have been reported,\nparticularly when there are multiple small adenomas. This presents an\nopportunity to leverage computer-aided systems to support clinicians and reduce\nthe number of polyps missed.\n Method: In this work we introduce the Focus U-Net, a novel dual\nattention-gated deep neural network, which combines efficient spatial and\nchannel-based attention into a single Focus Gate module to encourage selective\nlearning of polyp features. The Focus U-Net further incorporates short-range\nskip connections and deep supervision. Furthermore, we introduce the Hybrid\nFocal loss, a new compound loss function based on the Focal loss and Focal\nTversky loss, to handle class-imbalanced image segmentation. For our\nexperiments, we selected five public datasets containing images of polyps\nobtained during optical colonoscopy: CVC-ClinicDB, Kvasir-SEG, CVC-ColonDB,\nETIS-Larib PolypDB and EndoScene test set. To evaluate model performance, we\nuse the Dice similarity coefficient (DSC) and Intersection over Union (IoU)\nmetrics.\n Results: Our model achieves state-of-the-art results for both CVC-ClinicDB\nand Kvasir-SEG, with a mean DSC of 0.941 and 0.910, respectively. When\nevaluated on a combination of five public polyp datasets, our model similarly\nachieves state-of-the-art results with a mean DSC of 0.878 and mean IoU of\n0.809, a 14% and 15% improvement over the previous state-of-the-art results of\n0.768 and 0.702, respectively.\n Conclusions: This study shows the potential for deep learning to provide fast\nand accurate polyp segmentation results for use during colonoscopy. The Focus\nU-Net may be adapted for future use in newer non-invasive screening and more\nbroadly to other biomedical image segmentation tasks involving class imbalance\nand requiring efficiency.\n
BibTeX
@article{yeung2021focus,
  author = {Michael Yeung and Evis Sala and Carola‐Bibiane Schönlieb and Leonardo Rundo},
  title = {{Focus U-Net: A novel dual attention-gated CNN for polyp segmentation\textbackslash\{\}n during colonoscopy}},
  journal = {Computers in Biology and Medicine},
  year = {2021},
  doi = {10.1016/j.compbiomed.2021.104815},
}
Figure