TL;DR
Pathology foundation models (PFMs) excel at identifying clear-cut cancer but struggle with "atypia"—the ambiguous middle ground in breast pathology. This study reveals that slide-level classifiers fail not because they can't find the lesions, but because slide-level labels hide atypical regions and the models lack the quantitative "extent" measures required for clinical diagnosis.
What Problem Does This Paper Address?
In breast pathology, the line between a benign lesion (Usual Ductal Hyperplasia, UDH), an atypical one (Atypical Ductal Hyperplasia, ADH), and early-stage cancer (Ductal Carcinoma in Situ, DCIS) is incredibly thin. However, this boundary determines whether a patient undergoes surgery or simple monitoring.
Current computational pathology relies on Pathology Foundation Models (PFMs)—massive neural networks pre-trained on millions of images. While these models are great at spotting invasive carcinoma, they perform poorly on atypical lesions. The authors set out to diagnose why these models fail: Is it a lack of data? Are the labels wrong? Or is the model looking at the wrong thing?
Methods
The researchers didn't just build another classifier; they performed a "stress test" on four state-of-the-art PFMs: Virchow2, H-optimus-1, UNI2, and Prov-GigaPath.
Key methodological choices included:
- Multiple Instance Learning (MIL): Using an attention-based MIL aggregator to process whole-slide images (WSIs).
- Causal Probes: A novel "what-if" analysis where the researchers manually edited the "bag" of image patches—inserting atypical regions into benign slides or deleting diagnostic lesions—to see how the model’s probability scores shifted.
- Oracle Localization: Comparing slide-level performance against models given the exact location of the lesion (region-level labels) to separate the task of finding a lesion from the task of classifying it.
- Resolution Ladder: Testing the models at various magnifications (2.5x to 40x) to see if cytological details were being missed.
Datasets
The study utilized three distinct cohorts:
- BRACS: A public reference dataset for breast atypia (547 WSIs). This was the primary training and cross-validation set.
- External Hospital Cohort: 118 WSIs from a second hospital, specifically selected by pathologists to contain only a single lesion subtype per slide. This helped isolate the effect of "mixed" labels.
- HISTAI-breast: A public collection of 517 cases used for further external validation.
Key Insights and Ideas
- The "Maximum Rule" Labeling Trap: In standard datasets like BRACS, a slide is labeled by its most severe lesion. The authors found that only 39% of slides containing ADH were actually labeled as ADH; the rest were labeled as DCIS or IC because those lesions were also present. This means models are often trained on "atypia" that is effectively invisible to the loss function.
- Found, but not Identified: Attention mechanisms successfully "ranked" atypical lesions highly (finding them), but the classifiers still misidentified them. Simply showing the model the lesion (oracle localization) only provided a marginal boost, suggesting the problem lies in the classification logic, not the search.
- The Malignancy Bias: When ADH regions were inserted into benign slides, the models' probability for malignancy rose significantly more than their probability for atypia. The models seem to view atypia as "weak cancer" rather than a distinct biological category.
- Extent Matters: Clinically, the difference between ADH and low-grade DCIS is often a matter of extent (e.g., is the lesion larger than 2mm?). Slide-level models currently lack a mechanism to measure the total area or contiguity of suspicious patches, which is a hard requirement for these specific diagnostic boundaries.
- Domain Shift Collapse: When moving to new hospitals/scanners, triage specificity (the ability to correctly rule out benign cases) collapsed from ~50% to near 0%. Benign tissue from new sites looked "suspicious" enough to the models to trigger a false positive, likely due to differences in image acquisition.
Key Results
- Atypical Performance Gap: While models achieved high AUCs for malignant vs. rest (0.94–0.95), performance on atypical boundaries like UDH vs. ADH or FEA vs. ADH was much lower (0.52–0.58 at the slide level).
- Region-Level Gains: Using lesion-level labels and region-specific classifiers improved AUCs for all four atypical boundaries by +0.06 to +0.29, proving the information is in the image, but slide-level training fails to extract it.
- Triage Failure: A triage threshold that worked in cross-validation (ruling out half of benign slides) failed completely on external data, where benign slides were scored +0.41 to +0.97 standard deviations closer to lesions than in the training set.
Limitations and Future Work
The authors acknowledge that their external cohorts had relatively few atypical cases (e.g., only 15 atypical slides in the hospital cohort). Furthermore, the expert polygons used for "oracle" testing were representative rather than exhaustive.
The paper suggests that progress in atypia classification won't come from simply using larger foundation models. Instead, we need:
- Better Labels: Moving beyond the "maximum rule" to labels that account for all lesions present.
- Structural Aggregators: New MIL methods that can explicitly measure the extent and contiguity of lesions, mimicking how pathologists actually apply clinical criteria.
- Robustness to Acquisition: Better domain adaptation to prevent benign tissue from being misclassified due to scanner-specific artifacts.
Summary updated