CFPNet-M: A Light-Weight Encoder-Decoder Based Network for Multimodal Biomedical Image Real-Time Segmentation

Ange Lou, Shuyue Guan, Murray H. LoewComputers in Biology and Medicine 2021 · OpenReview

TL;DR

This paper introduces CFPNet-M, a novel, lightweight encoder-decoder network for real-time biomedical image segmentation. By incorporating a Channel-wise Feature Pyramid (CFP) module and simplifying the U-Net architecture, CFPNet-M achieves comparable segmentation accuracy to existing methods while drastically reducing model size and computational requirements, making it suitable for deployment on resource-constrained devices and real-time applications.

What Problem Does This Paper Address?

Medical image segmentation is crucial for diagnosis and treatment, with the U-Net architecture being a dominant approach. However, standard U-Net models suffer from several drawbacks: they are parameter-heavy, requiring significant computational resources and memory; they struggle to capture global context; and they can produce incomplete segmentations in challenging cases. Furthermore, the demand for real-time segmentation in applications like image-guided surgery and medical robotics necessitates models that are not only accurate but also efficient in terms of speed and size. This paper aims to bridge this gap by developing a lightweight, real-time segmentation network that maintains high accuracy across various medical imaging modalities.

Methods

The core innovation of this work lies in the Channel-wise Feature Pyramid (CFP) module and the overall CFPNet-M architecture.

The CFP module is built upon the concept of a Feature Pyramid (FP) channel, which decomposes large convolutional kernels into a series of smaller ones. The authors adapt this by:

  • Dilated Convolution: Incorporating dilated convolutions within the CFP module to expand the receptive field without increasing parameters, allowing for the capture of larger-scale features.
  • Factorization and Multi-Scale Features: Similar to Inception-v2, they use factorized convolutions (e.g., 3x3 to replace 5x5) to reduce parameters. They also design the module to capture multi-scale features using kernels of different sizes (3x3, 5x5, 7x7) and applying dilation rates.
  • Hierarchical Feature Fusion (HFF): To mitigate artifacts like gridding textures that can arise from simple fusion methods, they employ an HFF technique, inspired by prior work, which concatenates features from different convolutional operators.
  • Lightweight Design: The CFP module is designed to use significantly fewer parameters than traditional Inception-like structures while retaining the ability to learn multi-scale information.

The CFPNet-M architecture itself is a simplified U-Net. It uses the CFP module as a key component in its encoder path. The network is designed to be shallow, balancing depth and width to effectively capture both local and global features. The authors also introduce a novel measurement metric, the Tanimoto similarity, for comparing grayscale images, which they found to be more stable than traditional metrics like Jaccard similarity for this task.

Datasets

The authors evaluated CFPNet-M on five diverse medical imaging datasets:

  • Breast Thermography Images: Their own collected dataset.
  • Electron Microscopy Images: ISBI-2012 dataset.
  • Endoscopy Images: CVC-ClinicDB dataset.
  • Dermoscopy Images: ISIC-2018 dataset.
  • Digital Retinal Images: DRIVE dataset.

These datasets cover various modalities and object types, including region-based (tumors, lesions) and thin objects (vessels), to demonstrate the model's versatility.

Key Insights and Ideas

  • Lightweight is Achievable for High-Performance Segmentation: The paper demonstrates that it's possible to achieve competitive segmentation accuracy with drastically fewer parameters and smaller model sizes compared to established architectures like U-Net. This opens doors for deploying advanced segmentation models on edge devices or in real-time scenarios.
  • The Power of Dilated Convolutions and Feature Pyramids: The integration of dilated convolutions within a feature pyramid structure (CFP module) is key to capturing multi-scale information efficiently, a critical aspect for segmenting diverse medical image features.
  • Tanimoto Similarity as a Robust Metric: The introduction and validation of Tanimoto similarity as a preferred metric for grayscale image comparison in segmentation tasks is a valuable contribution, offering a more stable alternative to traditional binary metrics when dealing with continuous output values.
  • Real-time Segmentation for Medical Applications: The combination of high speed (80 FPS) and low computational cost makes CFPNet-M particularly promising for time-sensitive medical applications like robotic surgery and interactive diagnostic tools.

Key Results

  • Parameter Efficiency: CFPNet-M requires only 0.65 million parameters, which is approximately 2% of the original U-Net and significantly less than other state-of-the-art models like Inception v3 (about 2.2% of parameters). The model file size is also remarkably small at 8.8 MB.
  • Real-time Performance: The network achieves an impressive inference speed of 80 frames per second (FPS) on a single RTX 2070Ti GPU with an input size of 256x192 pixels.
  • Competitive Accuracy: CFPNet-M achieves segmentation results comparable to existing methods across all five tested medical datasets, often outperforming other lightweight models. For instance, on the IR breast dataset, it shows competitive accuracy with models like Inception v3.
  • Versatility: The model demonstrates strong performance on both region-based and thin object segmentation tasks, highlighting its adaptability to different image characteristics.

Limitations and Future Work

  • Limitations: While CFPNet-M shows excellent performance, the paper notes that for some challenging cases with small objects, vague boundaries, and unbalanced illumination (e.g., CVC-ClinicDB dataset), models like Inception v3, MobileNet v2, and EfficientNet-b0 might achieve slightly better segmentation results, though at a much higher computational cost. The authors also mention that they did not use additional data augmentation or enhancement techniques in their experiments, which could potentially further improve performance.
  • Future Work: The paper suggests that the lightweight nature and real-time capabilities of CFPNet-M make it suitable for deployment on portable devices and for enabling advanced medical applications such as image-guided surgery and vision-based medical robots. Further exploration into optimizing the architecture and exploring its application in more complex clinical scenarios would be logical next steps.

Summary updated

Abstract

Currently, developments of deep learning techniques are providing instrumental to identify, classify, and quantify patterns in medical images. Segmentation is one of the important applications in medical image analysis. In this regard, U-Net is the predominant approach to medical image segmentation tasks. However, we found that those U-Net based models have limitations in several aspects, for example, millions of parameters in the U-Net consuming considerable computation resource and memory, lack of global information, and missing some tough objects. Therefore, we applied two modifications to improve the U-Net model: 1) designed and added the dilated channel-wise CNN module, 2) simplified the U shape network. Based on these two modifications, we proposed a novel light-weight architecture -- Channel-wise Feature Pyramid Network for Medicine (CFPNet-M). To evaluate our method, we selected five datasets with different modalities: thermography, electron microscopy, endoscopy, dermoscopy, and digital retinal images. And we compared its performance with several models having different parameter scales. This paper also involves our previous studies of DC-UNet and some commonly used light-weight neural networks. We applied the Tanimoto similarity instead of the Jaccard index for gray-level image measurements. By comparison, CFPNet-M achieves comparable segmentation results on all five medical datasets with only 0.65 million parameters, which is about 2% of U-Net, and 8.8 MB memory. Meanwhile, the inference speed can reach 80 FPS on a single RTX 2070Ti GPU with the 256 by 192 pixels input size.
BibTeX
@article{lou2021cfpnet,
  author = {Ange Lou and Shuyue Guan and Murray H. Loew},
  title = {{CFPNet-M: A Light-Weight Encoder-Decoder Based Network for Multimodal Biomedical Image Real-Time Segmentation}},
  journal = {Computers in Biology and Medicine},
  year = {2021},
  url = {https://openreview.net/forum?id=Ft1agB0VQH},
}
Figure