Deformable Gabor Feature Networks for Biomedical Image Classification

Xuan Gong, Xin Xia, Wentao Zhu, Baochang Zhang, David S. Doermann, Li'an Zhuo

TL;DR

This paper introduces Deformable Gabor Feature Networks (DGFNs), a novel deep learning architecture that combines deformable convolutions with adaptive Gabor filters. DGFNs aim to improve the interpretability and robustness of medical image analysis by better capturing complex spatial variations, outperforming existing methods on mammogram and chest X-ray classification tasks.

What Problem Does This Paper Address?

Deep learning has revolutionized medical image analysis, but current Convolutional Neural Networks (CNNs) face limitations in representing the intricate geometric structures common in medical images. These limitations stem from two main issues: 1) the need for vast amounts of detailed, annotated data, which is often scarce in the medical domain, and 2) the "black-box" nature of CNNs, which hinders the exploitation of well-established domain-specific features. Existing methods struggle to effectively handle complex spatial variations like scale, orientation, and deformation, which are crucial for accurate diagnosis.

Methods

The core innovation is the Deformable Gabor Convolution (DGConv) module. This module integrates two key ideas:

  • Deformable Convolutions (DConv): Inspired by DCNs [3], DGConv learns local offsets for sampling feature maps. This allows the network to adapt its receptive field to the specific spatial variations of objects, making it more robust to deformations.
  • Adaptive Gabor Convolutions (GConv): Gabor filters are known for their ability to capture spatial locality, scale, and orientation selectivity, making them well-suited for image analysis. In DGConv, these Gabor filters are made adaptive by learning modulation masks. This allows the filters to adjust their characteristics based on the input data, enhancing their representativeness.

The DGConv module replaces standard convolutional layers, particularly in the higher-level feature extraction stages. The network, termed Deformable Gabor Feature Network (DGFN), is designed to be easily integrated into existing CNN architectures (like ResNet and DenseNet) with minimal additional parameters and training cost. The authors also propose a specific multi-instance learning framework to handle the weakly supervised nature of the datasets.

Datasets

The proposed DGFNs were evaluated on two publicly available medical imaging datasets:

  • INbreast Dataset: A dataset of mammograms used for mass malignancy classification. This is framed as a multi-instance learning problem.
  • ChestX-ray14 Dataset: A large dataset of chest X-ray images with multiple pathology labels. This is treated as a multi-instance, multi-label classification problem.

The paper also mentions experiments on the CIFAR-10 dataset (both clean and noisy versions) to demonstrate the general robustness of DGFNs to natural image variations and noise.

Key Insights and Ideas

  • Bridging Hand-crafted and Deep Features: The work effectively bridges the gap between traditional, interpretable hand-crafted features (like Gabor filters) and data-driven deep learning features. By making Gabor filters adaptive and integrating them with deformable convolutions, the network gains both interpretability and enhanced representational power.
  • Deformable Sampling + Adaptive Filters = Robustness: The combination of learning deformable sampling locations (DConv) and adaptive filter characteristics (GConv) is key. This dual approach allows the network to not only adapt its receptive field but also its feature extraction kernels to complex and varying spatial patterns.
  • Interpretability through Gabor Filters: While CNNs are often black boxes, the explicit use of Gabor filters, even in an adaptive and deformable manner, offers a potential pathway to better understanding what features the network is learning and why.
  • Efficiency and Compactness: The DGConv module is designed to be efficient, requiring only a small number of additional parameters compared to standard CNNs, making it practical for deployment.

Key Results

  • Improved Performance on Medical Datasets: DGFNs consistently outperformed baseline CNNs, Gabor Convolutional Networks (GCNs), and Deformable Convolutional Networks (DCNs) on both the INbreast and ChestX-ray14 datasets. For instance, on the INbreast-deform dataset, DGFNs achieved AUCs up to 83.30%, compared to DCNs at 82.03% and GCNs at 79.16% with similar model sizes.
  • Robustness to Deformation and Noise: Experiments on the INbreast-deform dataset and noisy CIFAR-10 demonstrated that DGFNs are significantly more robust to spatial variations and random noise than their counterparts.
  • Effective Feature Extraction: Visualizations on the INbreast dataset showed that DGFNs could effectively pinpoint malignant regions, highlighting the improved feature extraction capabilities.
  • Competitive Performance with Fewer Parameters: In some configurations, DGFNs achieved better performance with fewer parameters than comparable CNNs, indicating an efficient use of model capacity.

Limitations and Future Work

  • Computational Cost: While DGFNs are described as efficient, the paper notes that the number of learned masks (V) and orientation channels (U) can impact performance and complexity. The choice of these hyperparameters involves a trade-off.
  • Generalizability: While tested on natural images (CIFAR-10), the primary focus remains on medical imaging. Further validation on a wider range of natural image tasks could be beneficial.
  • Future Work: The authors suggest that the DGConv module is generic and flexible, implying its potential application to other deep learning architectures and tasks beyond classification. Further exploration of its use in segmentation or object detection tasks could be a natural next step. The paper also hints at further investigating the interpretability aspects of the learned Gabor filters.

Summary updated

Abstract

In recent years, deep learning has dominated progress in the field of medical image analysis. We find however, that the ability of current deep learning approaches to represent the complex geometric structures of many medical images is insufficient. One limitation is that deep learning models require a tremendous amount of data, and it is very difficult to obtain a sufficient amount with the necessary detail. A second limitation is that there are underlying features of these medical images that are well established, but the black-box nature of existing convolutional neural networks (CNNs) do not allow us to exploit them. In this paper, we revisit Gabor filters and introduce a deformable Gabor convolution (DGConv) to expand deep networks interpretability and enable complex spatial variations. The features are learned at deformable sampling locations with adaptive Gabor convolutions to improve representitiveness and robustness to complex objects. The DGConv replaces standard convolutional layers and is easily trained end-to-end, resulting in deformable Gabor feature network (DGFN) with few additional parameters and minimal additional training cost. We introduce DGFN for addressing deep multi-instance multi-label classification on the INbreast dataset for mammograms and on the ChestX-ray14 dataset for pulmonary x-ray images.
BibTeX
@inproceedings{gong2021deformable,
  author = {Xuan Gong and Xin Xia and Wentao Zhu and Baochang Zhang and David S. Doermann and Li'an Zhuo},
  title = {{Deformable Gabor Feature Networks for Biomedical Image Classification}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
  doi = {10.1109/wacv48630.2021.00405},
}
Figure