Automated Assessment of the Curliness of Collagen Fiber in Breast Cancer

David Paredes, Prateek Prasanna, Christina Preece, Rajarsi R. Gupta, Farzad Fereidouni, Dimitris Samaras, Tahsin Kurc, Richard M. Levenson, Patricia Thompson-Carino, Joel Saltz, Chao ChenBIC @ ECCV 2020 · OpenReview

TL;DR

This paper introduces a novel deep learning framework to automatically classify collagen fiber patterns as "curly" or "straight" in breast cancer tissue images. By combining traditional image processing techniques (ridge detection) with deep neural networks, they achieve improved accuracy in identifying these structural features, which are linked to tumor progression and metastasis.

What Problem Does This Paper Address?

The extracellular matrix (ECM) in tumors, particularly the collagen fibers within it, plays a crucial role in breast cancer progression and metastasis. Specifically, the alignment and structure of collagen fibers have been correlated with tumor cell migration and invasion. While straight collagen alignment is associated with increased migration, understanding the "curliness" of collagen could offer valuable prognostic insights. However, existing methods for characterizing collagen structure are often expensive, require specialized equipment, or suffer from inconsistencies in clinical settings. There's a clear need for an automated, reliable, and accessible method to quantify collagen fiber architecture, specifically its curliness, to aid in diagnosis and prognosis.

Methods

The core of this work lies in a two-stream deep learning approach that leverages both the raw image data and enhanced features derived from it.

  1. Ridge Detection: The authors employ a Hessian-based ridge detector, a classic image processing technique, to highlight the curvilinear structures of collagen fibers. This method identifies ridges by analyzing the eigenvalues of the Hessian matrix, effectively capturing the local orientation and curvature of the fibers. This step is crucial for emphasizing the characteristic fiber gradients.
  2. Two-Stream Convolutional Neural Networks (CNNs): They propose a two-stream architecture. One stream processes the original DUET image patches, while the other stream processes the output of the ridge detector. Both streams utilize pre-trained ResNet models (ImageNet weights) with their final layers fine-tuned for the binary classification task (curly vs. straight).
  3. Feature Fusion: To combine the information from both streams, they explore several feature fusion strategies: element-wise product, addition, concatenation, and a bilinear operation. This fusion aims to exploit complementary information from the raw image and the ridge-filtered features.

Datasets

The study utilizes a unique dataset of DUET (DUal-mode Emission and Transmission) images of breast cancer (BCa) tissue. These images are derived from hematoxylin and eosin (H&E) stained tissue samples.

  • Dataset Origin: Six whole slide images (WSIs) from patients with aggressive early-stage breast cancer were used.
  • Annotation: Regions of interest (ROIs) with strong collagen structures were selected by a domain expert and given preliminary "curly" or "straight" labels.
  • Patch Sampling: Non-overlapping patches of 400x400 pixels (0.22 microns/pixel resolution) were sampled from these ROIs.
  • Patch Labels: Patches were further annotated as "curly," "straight," "mixed," or "insufficient signal." "Mixed" and "insufficient signal" patches were discarded.
  • Training/Validation Split: 5 WSIs were used for training and 1 WSI for validation.

Key Insights and Ideas

  • Ridge Detection as a Preprocessing Step: The paper demonstrates that enhancing collagen fiber structures using ridge detection before feeding them into a CNN significantly improves classification performance compared to using raw patches alone. This highlights the power of incorporating domain-specific image processing for feature extraction.
  • Two-Stream Networks for Structural Analysis: The two-stream architecture, processing both raw and enhanced (ridge-detected) features, proves effective. This suggests that combining different representations of the same underlying structure can capture more nuanced information than a single representation.
  • Feature Fusion Matters: While not all fusion strategies were equally effective, the "addition" method showed a notable boost in AUC, indicating that intelligently combining features from different modalities can lead to better generalization.
  • Clinical Relevance of Collagen Structure: The work reinforces the growing understanding that collagen architecture is not just structural but actively influences cancer progression, making its automated analysis a promising avenue for digital pathology.

Key Results

  • Improved AUC with Ridge Detector: Networks trained with ridge detector outputs achieved higher Area Under the ROC Curve (AUC) than those trained directly on raw patches. For example, the average AUC for raw patches was 0.7489, while with ridge filter output, it increased to 0.8372 (using a sigma of 5).
  • Feature Fusion Benefits: Among the fusion strategies, the "addition" method showed the most consistent improvement in AUC across different ResNet architectures when combining raw and ridge features. The average AUC for this fusion strategy was 0.8415.
  • Qualitative Validation: The predicted "curliness" heatmaps showed high consistency with the preliminary annotations of ROIs, visually confirming the model's ability to capture the desired structural patterns.

Limitations and Future Work

  • Limited Training Data: The authors acknowledge that the amount of training data is relatively limited, which might hinder the performance of deep neural networks trained from scratch. They suspect that with a larger cohort, feature fusion methods would become even more advantageous.
  • Generalizability: While the study uses a unique DUET dataset, further validation on larger and more diverse datasets would be beneficial to confirm the generalizability of the approach.
  • Future Directions: The authors suggest that this approach can be extended to other digital pathology analyses, such as automated tumor and TIL detection, and characterization of different immunohistochemical (IHC) stains. The framework could also be integrated with other types of image analysis to provide a more comprehensive understanding of tissue samples.

Summary updated

Abstract

The growth and spread of breast cancer tumor are influenced by the composition and structural properties of collagen in the extracellular matrix of tumors. Straight alignment of collagen has been attributed to tumor cell migration, which is correlated with tumor progression and metastasis in breast cancer. Thus, there is a need to characterize collagen alignment to study its value as a prognostic biomarker. We present a framework to characterize the curliness of collagen fibers in breast cancer images from DUET (DUal-mode Emission and Transmission) studies on hematoxylin and eosin (H\E) stained tissue samples. Our novel approach highlights the characteristic fiber gradients using a standard ridge detection method before feeding into the convolutional neural network. Experiments were performed on patches of breast cancer images containing straight or curly collagen. The proposed approach outperforms in terms of area under the curve against transfer learning methods trained directly on the original patches. Furthermore, a two-stream network boosts the performance by exploiting the original patches and their ridge filter responses.
BibTeX
@inproceedings{paredes2020automated,
  author = {David Paredes and Prateek Prasanna and Christina Preece and Rajarsi R. Gupta and Farzad Fereidouni and Dimitris Samaras and Tahsin Kurc and Richard M. Levenson and Patricia Thompson-Carino and Joel Saltz and Chao Chen},
  title = {{Automated Assessment of the Curliness of Collagen Fiber in Breast Cancer}},
  booktitle = {BIC @ ECCV},
  year = {2020},
  url = {https://openreview.net/forum?id=6wlFgoN1h7b},
}
Figure