Where slide-level foundation-model classifiers fail on atypical breast lesions

Nerea Hernandez, Luis J. Herrera, Cesar Ramirez, Olga Valenzuela, Ignacio Rojas, Francisco M. OrtuñoarXiv 2026 · arXiv

TL;DR

Pathology foundation models (PFMs) excel at identifying clear-cut cancer but struggle with "atypia"—the ambiguous middle ground in breast pathology. This study reveals that slide-level classifiers fail not because they can't find the lesions, but because slide-level labels hide atypical regions and the models lack the quantitative "extent" measures required for clinical diagnosis.

What Problem Does This Paper Address?

In breast pathology, the line between a benign lesion (Usual Ductal Hyperplasia, UDH), an atypical one (Atypical Ductal Hyperplasia, ADH), and early-stage cancer (Ductal Carcinoma in Situ, DCIS) is incredibly thin. However, this boundary determines whether a patient undergoes surgery or simple monitoring.

Current computational pathology relies on Pathology Foundation Models (PFMs)—massive neural networks pre-trained on millions of images. While these models are great at spotting invasive carcinoma, they perform poorly on atypical lesions. The authors set out to diagnose why these models fail: Is it a lack of data? Are the labels wrong? Or is the model looking at the wrong thing?

Methods

The researchers didn't just build another classifier; they performed a "stress test" on four state-of-the-art PFMs: Virchow2, H-optimus-1, UNI2, and Prov-GigaPath.

Key methodological choices included:

  • Multiple Instance Learning (MIL): Using an attention-based MIL aggregator to process whole-slide images (WSIs).
  • Causal Probes: A novel "what-if" analysis where the researchers manually edited the "bag" of image patches—inserting atypical regions into benign slides or deleting diagnostic lesions—to see how the model’s probability scores shifted.
  • Oracle Localization: Comparing slide-level performance against models given the exact location of the lesion (region-level labels) to separate the task of finding a lesion from the task of classifying it.
  • Resolution Ladder: Testing the models at various magnifications (2.5x to 40x) to see if cytological details were being missed.

Datasets

The study utilized three distinct cohorts:

  1. BRACS: A public reference dataset for breast atypia (547 WSIs). This was the primary training and cross-validation set.
  2. External Hospital Cohort: 118 WSIs from a second hospital, specifically selected by pathologists to contain only a single lesion subtype per slide. This helped isolate the effect of "mixed" labels.
  3. HISTAI-breast: A public collection of 517 cases used for further external validation.

Key Insights and Ideas

  • The "Maximum Rule" Labeling Trap: In standard datasets like BRACS, a slide is labeled by its most severe lesion. The authors found that only 39% of slides containing ADH were actually labeled as ADH; the rest were labeled as DCIS or IC because those lesions were also present. This means models are often trained on "atypia" that is effectively invisible to the loss function.
  • Found, but not Identified: Attention mechanisms successfully "ranked" atypical lesions highly (finding them), but the classifiers still misidentified them. Simply showing the model the lesion (oracle localization) only provided a marginal boost, suggesting the problem lies in the classification logic, not the search.
  • The Malignancy Bias: When ADH regions were inserted into benign slides, the models' probability for malignancy rose significantly more than their probability for atypia. The models seem to view atypia as "weak cancer" rather than a distinct biological category.
  • Extent Matters: Clinically, the difference between ADH and low-grade DCIS is often a matter of extent (e.g., is the lesion larger than 2mm?). Slide-level models currently lack a mechanism to measure the total area or contiguity of suspicious patches, which is a hard requirement for these specific diagnostic boundaries.
  • Domain Shift Collapse: When moving to new hospitals/scanners, triage specificity (the ability to correctly rule out benign cases) collapsed from ~50% to near 0%. Benign tissue from new sites looked "suspicious" enough to the models to trigger a false positive, likely due to differences in image acquisition.

Key Results

  • Atypical Performance Gap: While models achieved high AUCs for malignant vs. rest (0.94–0.95), performance on atypical boundaries like UDH vs. ADH or FEA vs. ADH was much lower (0.52–0.58 at the slide level).
  • Region-Level Gains: Using lesion-level labels and region-specific classifiers improved AUCs for all four atypical boundaries by +0.06 to +0.29, proving the information is in the image, but slide-level training fails to extract it.
  • Triage Failure: A triage threshold that worked in cross-validation (ruling out half of benign slides) failed completely on external data, where benign slides were scored +0.41 to +0.97 standard deviations closer to lesions than in the training set.

Limitations and Future Work

The authors acknowledge that their external cohorts had relatively few atypical cases (e.g., only 15 atypical slides in the hospital cohort). Furthermore, the expert polygons used for "oracle" testing were representative rather than exhaustive.

The paper suggests that progress in atypia classification won't come from simply using larger foundation models. Instead, we need:

  1. Better Labels: Moving beyond the "maximum rule" to labels that account for all lesions present.
  2. Structural Aggregators: New MIL methods that can explicitly measure the extent and contiguity of lesions, mimicking how pathologists actually apply clinical criteria.
  3. Robustness to Acquisition: Better domain adaptation to prevent benign tissue from being misclassified due to scanner-specific artifacts.

Summary updated

Abstract

Whether a ductal breast lesion is called atypical hyperplasia or low-grade carcinoma in situ increasingly determines surgery. However, slide-level classifiers built on pathology foundation models (PFMs) perform worst on these lesions. Using four PFMs, cross-validation on BRACS and preregistered hypotheses, we studied where the classification of atypia fails. Because slides are labelled with their most severe lesion, only 39% of the slides with annotated atypical ductal hyperplasia (ADH) were labelled ADH. Attention ranked the lesion high but rarely first, and showing the model only the lesion helped little, whereas classifiers trained per decision with lesion-level labels improved all four atypical boundaries. ADH regions inserted into benign slides raised the probability of malignancy more than that of atypia. In a hospital with pathologist-labelled single-subtype slides and in HISTAI, benign slides were scored closer to lesions and triage specificity fell from 0.48-0.53 to 0.00-0.07 and 0.22.
BibTeX
@misc{hernandez2026where,
  author = {Nerea Hernandez and Luis J. Herrera and Cesar Ramirez and Olga Valenzuela and Ignacio Rojas and Francisco M. Ortuño},
  title = {{Where slide-level foundation-model classifiers fail on atypical breast lesions}},
  year = {2026},
  eprint = {2611.19630},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19630},
}
Figure