Centroid-Aware Feature Recalibration for Cancer Grading in Pathology Images

Jaeung Lee, Keunho Byeon, Jin Tae Kwak

TL;DR

This paper introduces CaFeNet, a novel deep learning architecture for cancer grading in pathology images. CaFeNet enhances feature representations by recalibrating them using class centroids via an attention mechanism, leading to more accurate and robust cancer grading, even across datasets with varying environmental conditions.

What Problem Does This Paper Address?

Accurate cancer grading is crucial for determining patient treatment and prognosis. Traditionally, this is a manual and time-consuming process performed by pathologists, which can lead to errors and delays due to workload. While deep learning, specifically deep convolutional neural networks (DCNNs), has shown promise in automating this task, existing methods still struggle with robustness and reliability, particularly when faced with variations in image acquisition environments. This paper aims to address this by developing a more robust and accurate automated cancer grading system.

Methods

The core innovation of CaFeNet lies in its "centroid-aware feature recalibration" mechanism. The architecture consists of three main components:

  1. Feature Extractor: An EfficientNet-B0 backbone is used to extract initial feature representations from pathology images.
  2. Centroid Update (Cup) Module: This module computes and updates the "centroid embedding vectors" for each cancer grade. These centroids represent the average feature representation for each class in the embedding space.
  3. Centroid-Aware Feature Recalibration (CaFe) Module: This is the key novel component. It takes the initial embedding vector of an input image and recalibrates it by using an attention mechanism. The attention mechanism learns to weigh the input embedding vector based on its similarity to the pre-computed class centroid embedding vectors. This recalibration aims to align the image's features with the typical representation of its class, making the final representation more stable and discriminative.

The recalibrated embedding vector is then concatenated with the original embedding vector and fed into a classification layer for the final cancer grade prediction.

Datasets

The study evaluated CaFeNet on two publicly available colorectal cancer datasets:

  • Dataset 1 (C<sub>train</sub>, C<sub>validation</sub>, C<sub>test1</sub>): This dataset consists of 1600 benign, 2322 well-differentiated (WD), 4105 moderately-differentiated (MD), and 1830 poorly-differentiated (PD) image patches. These were collected between 2006-2008 using an Aperio digital slide scanner at 40x magnification, with image patches of size 1024x1024 pixels.
  • Dataset 2 (C<sub>testII</sub>): This dataset contains 27986 benign, 8394 WD, 61985 MD, and 11985 PD image patches of size 1144x1144 pixels. These were acquired between 2016-2017 using a NanoZoomer digital slide scanner.

The C<sub>train</sub> and C<sub>validation</sub> sets were used for training and hyperparameter tuning, while C<sub>test1</sub> and C<sub>testII</sub> were used for evaluation. Notably, C<sub>testII</sub> was acquired under different conditions (time period, scanner) to test the model's generalizability.

Key Insights and Ideas

  • Centroids as References: The paper posits that class centroids in the embedding space can serve as reliable reference points for data distribution, even when the input data distribution changes.
  • Attention for Recalibration: Using an attention mechanism to recalibrate image features based on their proximity to class centroids is a novel approach for improving robustness. It allows the model to dynamically adjust feature representations to better align with class characteristics.
  • Robustness to Environmental Changes: The core hypothesis is that by recalibrating features against stable class centroids, the model becomes less sensitive to variations in image acquisition, leading to more consistent performance across different datasets.
  • Concatenation for Feature Fusion: The study found that concatenating the original and recalibrated embedding vectors yielded better results than simply adding them, suggesting that both original and refined features contribute valuable information.

Key Results

CaFeNet demonstrated strong performance on colorectal cancer grading.

  • On the C<sub>test1</sub> dataset, CaFeNet achieved an accuracy of 87.5%, outperforming most competing models, including ResNet, DenseNet, EfficientNet, Triplet, SC, ViT, and Swin, and was comparable to the state-of-the-art MMAE-CEO model.
  • Crucially, on the C<sub>testII</sub> dataset (collected under different conditions), CaFeNet achieved the highest accuracy of 82.7%, significantly outperforming other models and showing a minimal performance drop compared to its performance on C<sub>test1</sub>. This highlights its superior generalizability.
  • Ablation studies confirmed the effectiveness of the CaFe module, showing a substantial performance drop when it was excluded.

Limitations and Future Work

The authors acknowledge the following limitations:

  • Limited Dataset Scope: The experiments were conducted on only two colorectal cancer datasets from a single institute.
  • Future Work: The authors plan to:
  • Conduct additional experiments on more diverse colorectal cancer datasets.
  • Validate the effectiveness of CaFeNet for other types of cancers and tissues in pathology images.
  • Further explore the generalizability of the proposed network.

Summary updated

Abstract

Cancer grading is an essential task in pathology. The recent developments of artificial neural networks in computational pathology have shown that these methods hold great potential for improving the accuracy and quality of cancer diagnosis. However, the issues with the robustness and reliability of such methods have not been fully resolved yet. Herein, we propose a centroid-aware feature recalibration network that can conduct cancer grading in an accurate and robust manner. The proposed network maps an input pathology image into an embedding space and adjusts it by using centroids embedding vectors of different cancer grades via attention mechanism. Equipped with the recalibrated embedding vector, the proposed network classifiers the input pathology image into a pertinent class label, i.e., cancer grade. We evaluate the proposed network using colorectal cancer datasets that were collected under different environments. The experimental results confirm that the proposed network is able to conduct cancer grading in pathology images with high accuracy regardless of the environmental changes in the datasets.
BibTeX
@inproceedings{lee2023centroid,
  author = {Jaeung Lee and Keunho Byeon and Jin Tae Kwak},
  title = {{Centroid-Aware Feature Recalibration for Cancer Grading in Pathology Images}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2023},
  url = {https://openreview.net/forum?id=ezGNNeb3Tu},
}
Figure