Capsule Networks for Brain Tumor Classification Based on MRI Images and Coarse Tumor Boundaries

Parnian Afshar, Konstantinos N. Plataniotis, Arash Mohammadi

TL;DR

This paper introduces a modified Capsule Network (CapsNet) architecture for brain tumor classification that leverages both raw MRI images and coarse tumor boundaries. By incorporating these boundaries, the model achieves higher accuracy than previous methods, addressing the limitations of standard Convolutional Neural Networks (CNNs) in capturing spatial relationships and the sensitivity of CapsNets to image backgrounds.

What Problem Does This Paper Address?

Brain tumors are a deadly form of cancer, and accurately classifying their type is crucial for effective treatment and improving patient survival rates. While human diagnosis is prone to errors, automated methods using Convolutional Neural Networks (CNNs) have emerged. However, CNNs struggle to fully utilize spatial relationships between the tumor and surrounding tissues, which are vital for classification. Capsule Networks (CapsNets) offer a promising alternative due to their inherent ability to preserve spatial information. Yet, standard CapsNets are highly sensitive to image backgrounds, leading to reduced accuracy when processing whole brain images compared to segmented tumor regions. Furthermore, obtaining precise tumor segmentations is a time-consuming task requiring expert knowledge. This paper aims to bridge this gap by developing a CapsNet that can effectively use both raw brain MRI images and coarse tumor boundaries, improving classification accuracy without demanding precise tumor annotations.

Methods

The core of this work is a modified CapsNet architecture. The authors build upon the "Routing by Agreement" mechanism inherent in CapsNets, which allows capsules to predict and agree on parent capsule outputs, thus preserving spatial relationships. To address the background sensitivity of CapsNets and incorporate crucial contextual information from surrounding tissues, the proposed architecture takes the tumor's coarse boundaries as an additional input. This boundary information is concatenated with the output of the capsule layer before being fed into fully connected layers and a final Softmax layer for classification. This approach allows the CapsNet to maintain focus on the tumor while still benefiting from the surrounding tissue context, without requiring precise segmentation. The loss function is a combination of the standard CapsNet loss and a cross-entropy loss for the Softmax layer, with the CapsNet loss weighted more heavily.

Datasets

The study utilizes a brain MRI dataset consisting of 3064 images from 233 patients diagnosed with one of three brain tumor types: Meningioma, Pituitary, and Glioma. This dataset was used for both training and evaluation.

Key Insights and Ideas

  • Coarse Boundaries as a "Focusing Lens": The key insight is that providing coarse tumor boundaries to a CapsNet acts like a "focusing lens," guiding the network's attention to the relevant tumor region while still allowing it to consider the surrounding context. This is more effective than relying solely on segmented tumors or raw images.
  • Balancing Context and Specificity: The proposed method strikes a balance between the detailed spatial information captured by CapsNets and the need to avoid distraction from irrelevant background noise.
  • Reducing Annotation Burden: By using coarse boundaries instead of precise segmentations, the method significantly reduces the need for expert annotation, making the classification process more efficient and accessible.
  • CapsNets' Advantage in Spatial Reasoning: The paper reinforces the idea that CapsNets are inherently better suited for tasks requiring robust spatial reasoning compared to traditional CNNs, especially when dealing with transformations or variations in object orientation.

Key Results

The proposed modified CapsNet architecture achieved a classification accuracy of 90.89%. This significantly outperforms:

  • A standard CapsNet using only brain images (78% accuracy).
  • A standard CapsNet using only segmented tumor images (86.56% accuracy).
  • A CNN using only brain images (61.97% accuracy).
  • A CNN using only segmented tumor images (72.13% accuracy).
  • A modified CNN that also incorporates brain images and bounding boxes (88.33% accuracy).

The paper highlights that the proposed CapsNet architecture achieves the best performance when fed with both brain images and coarse tumor boundaries.

Limitations and Future Work

The authors acknowledge that while their approach eliminates the need for exact tumor annotation, it still relies on coarse boundaries. Future work will focus on investigating the interpretability of CapsNets in the context of medical decision-making, as the output instantiation parameters of the true class can potentially explain whether the network has captured correct features. This interpretability is considered a significant advantage of CapsNets in medical applications.

Summary updated

Abstract

According to official statistics, cancer is considered as the second leading cause of human fatalities. Among different types of cancer, brain tumor is seen as one of the deadliest forms due to its aggressive nature, heterogeneous characteristics, and low relative survival rate. Determining the type of brain tumor has significant impact on the treatment choice and patient's survival. Human-centered diagnosis is typically error-prone and unreliable resulting in a recent surge of interest to automatize this process using convolutional neural networks (CNNs). CNNs, however, fail to fully utilize spatial relations, which is particularly harmful for tumor classification, as the relation between the tumor and its surrounding tissue is a critical indicator of the tumor's type. In our recent work, we have incorporated newly developed CapsNets to overcome this shortcoming. CapsNets are, however, highly sensitive to the miscellaneous image background. The paper addresses this gap. The main contribution is to equip CapsNet with access to the tumor surrounding tissues, without distracting it from the main target. A modified CapsNet architecture is, therefore, proposed for brain tumor classification, which takes the tumor coarse boundaries as extra inputs within its pipeline to increase the CapsNet's focus. The proposed approach noticeably outperforms its counterparts.
BibTeX
@inproceedings{afshar2019capsule,
  author = {Parnian Afshar and Konstantinos N. Plataniotis and Arash Mohammadi},
  title = {{Capsule Networks for Brain Tumor Classification Based on MRI Images and Coarse Tumor Boundaries}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=SGRh5pgt-xd},
}

Loading related content...

Figure