Deep learning-based assessment of tumor-associated stroma for diagnosing breast cancer in histopathology images

Babak Ehteshami Bejnordi, Jimmy Lin, Ben Glass, Maeve Mullooly, Gretchen L. Gierach, Mark E. Sherman, Nico Karssemeijer, Jeroen van der Laak, Andrew H. Beck

TL;DR

This paper presents a deep learning framework that diagnoses breast cancer by shifting the focus from cancer cells (epithelium) to the surrounding supportive tissue, known as the stroma. By using a multi-stage convolutional neural network (CNN) approach to analyze "tumor-associated stroma," the researchers achieved a high diagnostic accuracy (AUC of 0.92), proving that the environment surrounding a tumor is a powerful, yet often neglected, diagnostic biomarker.

What Problem Does This Paper Address?

In the world of pathology, the "gold standard" for breast cancer diagnosis has traditionally focused on the "seeds": the epithelial cells. Pathologists look for abnormalities in cell shape, size, and how they form structures like tubules. Naturally, most computer-aided design (CAD) systems have followed suit, focusing their algorithms on segmenting nuclei and glands.

However, cancer doesn't exist in a vacuum. It relies on the "soil"—the stroma—which consists of connective tissue, blood vessels, and inflammatory cells. Recent biological research suggests that "cancer-associated fibroblasts" in the stroma actually drive tumor progression. The problem is that there is no precise, universally agreed-upon definition of what "cancer-associated stroma" looks like under a microscope, making it nearly impossible to design traditional "hand-crafted" algorithms to detect it. This paper addresses that gap by using deep learning to discover these subtle stromal patterns automatically.

Methods

The authors developed a multi-step pipeline to process Whole-Slide Images (WSIs):

  1. Tissue Segmentation (CNN I): A VGG-like network with 11 layers classifies the image into three broad categories: epithelium, stroma, and fat. This network was modified to be "fully convolutional," allowing it to process images of any size.
  2. Stromal Classification (CNN II): A deeper, 16-layer VGG-net focuses specifically on the stromal regions identified in the first step. It classifies them as either "normal stroma" or "tumor-associated stroma."
  3. Feature Extraction: The researchers extracted 71 distinct features from the outputs of both CNNs. These included:
  4. Global tissue amounts: Total area of each tissue type.
  5. Morphology: The shape and eccentricity of epithelial regions.
  6. Spatial Architecture: Using Delaunay Triangulation and Voronoi diagrams to map the "neighborhood" and distribution of epithelial regions.
  7. Final Classification: These 71 features were fed into a Random Forest classifier to make the final call: cancer or non-cancer.

To handle the fact that there is much more normal stroma than tumor-associated stroma in a typical slide, they implemented a "gradually increasing misclassification loss," which forced the model to become more sensitive to the rare tumor-associated regions over time.

Datasets

The study utilized a substantial dataset of 646 H&E stained breast tissue biopsies from 444 women, sourced from the BREAST Stamp Project (2007-2010).

  • Training/Validation: 350 WSIs.
  • Independent Testing: 296 WSIs (251 benign, 45 invasive cancer).
  • Annotations: Ground truth was established through manual annotations by trained students (for general tissue types) and supervised by an expert pathologist (specifically for tumor-associated stroma).

Key Insights and Ideas

  • The Stroma is a Stronger Signal: One of the most surprising findings was that features derived from the stroma-specific network (CNN II) were actually more discriminative than those from the epithelium-focused network (CNN I).
  • Beyond the Pixel: The paper doesn't just look at pixel intensity; it uses geometric constructs like Voronoi zones to capture the "zone of influence" of epithelial regions, effectively modeling the spatial relationship between the "seeds" and the "soil."
  • Data-Driven Discovery: Because pathologists struggle to define the exact morphology of cancer-associated stroma, the CNN acts as a feature discovery tool, identifying statistical variations in the tissue that humans might overlook.

Key Results

The system demonstrated impressive diagnostic power on the independent test set:

  • Overall Performance: The combined system achieved an Area Under the ROC Curve (AUC) of 0.921.
  • Stroma vs. Epithelium: When used in isolation, stromal features (CNN II) achieved an AUC of 0.904, while epithelial features (CNN I) achieved an AUC of 0.882.
  • Pixel Accuracy: CNN I achieved 95.5% accuracy in tissue segmentation, and CNN II achieved 92.0% accuracy in distinguishing normal from tumor-associated stroma.

These results confirm that the stroma contains sufficient information to accurately diagnose the presence of invasive cancer, even without a primary focus on the cancer cells themselves.

Limitations and Future Work

  • Diagnostic vs. Prognostic: While this study proves the stroma is great for diagnosis (is cancer present?), the authors suggest the next frontier is prognosis (will the cancer return?).
  • Benign Risk Assessment: A promising future direction is determining if "cancer-like" stroma in a currently benign biopsy could predict a higher risk of developing cancer in the future.
  • Biological Interpretability: While the CNN is effective, it remains a "black box." Future work could involve "peeking" into the network to see exactly which stromal textures or patterns it has identified as "tumor-associated," potentially providing new biological insights for pathologists.

Summary updated

Abstract

Diagnosis of breast carcinomas has so far been limited to the morphological interpretation of epithelial cells and the assessment of epithelial tissue architecture. Consequently, most of the automated systems have focused on characterizing the epithelial regions of the breast to detect cancer. In this paper, we propose a system for classification of hematoxylin and eosin (H&E) stained breast specimens based on convolutional neural networks that primarily targets the assessment of tumor-associated stroma to diagnose breast cancer patients. We evaluate the performance of our proposed system using a large cohort containing 646 breast tissue biopsies. Our evaluations show that the proposed system achieves an area under ROC of 0.92, demonstrating the discriminative power of previously neglected tumor-associated stroma as a diagnostic biomarker.
BibTeX
@inproceedings{bejnordi2017deep,
  author = {Babak Ehteshami Bejnordi and Jimmy Lin and Ben Glass and Maeve Mullooly and Gretchen L. Gierach and Mark E. Sherman and Nico Karssemeijer and Jeroen van der Laak and Andrew H. Beck},
  title = {{Deep learning-based assessment of tumor-associated stroma for diagnosing breast cancer in histopathology images}},
  booktitle = {ISBI},
  year = {2017},
  url = {https://openreview.net/forum?id=cTOLcFPuCo},
}

Loading related content...

Figure