Deep Interpretable Classification and Weakly-Supervised Segmentation of Histology Images via Max-Min Uncertainty

Soufiane Belharbi, Jérôme Rony, Jose Dolz, Ismail Ben Ayed, Luke McCaffrey, Eric Granger

Introduction

In medical imaging, particularly histology, obtaining pixel-level annotations for training segmentation models is labor-intensive and expensive. Weakly-supervised learning (WSL) offers a solution by using only image-level labels (e.g., "cancerous" vs. "healthy"). However, standard WSL methods often suffer from over-segmentation, incorrectly identifying healthy tissue as diseased because they lack a formal mechanism to model the background. This paper introduces a "Max-Min Uncertainty" framework to address these challenges by forcing models to be highly confident about foreground regions and highly uncertain about background regions.

The Max-Min Uncertainty Framework

The core innovation of this work is the decomposition of the input image into two distinct regions: a decidable foreground ($X^+$) and an undecidable background ($X^-$).

Decomposition of input image into decidable foreground and undecidable background regions.

By explicitly modeling what the background should look like to a classifier—specifically, that it should contain no discriminative information—the framework significantly reduces false positives and enhances the interpretability of the resulting segmentation masks.

Methodology

The proposed architecture utilizes a dual-branch system consisting of a Localizer ($M$) and a Classifier ($C$).

The proposed dual-branch architecture for training and inference.

Training Objectives

  1. Foreground Localization ($X^+$): The localizer identifies a region where the classifier should be very certain. The model minimizes standard cross-entropy here, pushing the prediction toward the ground-truth global label.
  2. Background Localization ($X^-$): This is the "Max" part of the uncertainty. The localizer identifies a region where the classifier should be maximally uncertain. The authors use Kullback-Leibler (KL) divergence to force the classifier’s output for the background to match a uniform distribution.
  3. Absolute Size Constraints (ASC): To prevent the model from selecting the entire image as foreground (a common "trivial solution" in WSL), the authors implement log-barrier regularization. These terms penalize extremely imbalanced segmentations, encouraging the model to find the largest reasonable regions for both foreground and background without requiring exact prior knowledge of object sizes.

Experimental Results

The framework was evaluated on two public histology benchmarks: the GlaS (Gland Segmentation) dataset and the Camelyon16 patch-based benchmark.

  • GlaS Dataset: The model achieved a 72.11% F1+ score for foreground segmentation, a 3.5% improvement over the next best method. More significantly, the F1- score (background accuracy) jumped from ~41% in previous methods to ~69%, indicating a massive reduction in false positives. The model also achieved a 0% classification error on the test set.
  • Camelyon16: The model maintained competitive classification accuracy while providing much cleaner segmentation masks, especially in "normal" patches where other models frequently hallucinated tumors.

Conclusion

The Max-Min Uncertainty framework provides a robust, interpretable method for weakly-supervised histology analysis. By treating the absence of evidence as a defining characteristic of the background, the model offers a "white-box" tool that pathologists can trust. While currently focused on binary classification, the success of the uncertainty-based masking suggests a promising path forward for multi-class medical image analysis.

Summary updated

Abstract

Weakly-supervised learning (WSL) has recently triggered substantial interest as it mitigates the lack of pixel-wise annotations. Given global image labels, WSL methods yield pixel-level predictions (segmentations), which enable to interpret class predictions. Despite their recent success, mostly with natural images, such methods can face important challenges when the foreground and background regions have similar visual cues, yielding high false-positive rates in segmentations, as is the case in challenging histology images. WSL training is commonly driven by standard classification losses, which implicitly maximize model confidence, and locate the discriminative regions linked to classification decisions. Therefore, they lack mechanisms for modeling explicitly non-discriminative regions and reducing false-positive rates. We propose novel regularization terms, which enable the model to seek both non-discriminative and discriminative regions, while discouraging unbalanced segmentations. We introduce high uncertainty as a criterion to localize non-discriminative regions that do not affect classifier decision, and describe it with original Kullback-Leibler (KL) divergence losses evaluating the deviation of posterior predictions from the uniform distribution. Our KL terms encourage high uncertainty of the model when the latter inputs the latent non-discriminative regions. Our loss integrates: (i) a cross-entropy seeking a foreground, where model confidence about class prediction is high; (ii) a KL regularizer seeking a background, where model uncertainty is high; and (iii) log-barrier terms discouraging unbalanced segmentations. Comprehensive experiments and ablation studies over the public GlaS colon cancer data and a Camelyon16 patch-based benchmark for breast cancer show substantial improvements over state-of-the-art WSL methods, and confirm the effect of our new regularizers.
BibTeX
@article{belharbi2022deep,
  author = {Soufiane Belharbi and Jérôme Rony and Jose Dolz and Ismail Ben Ayed and Luke McCaffrey and Eric Granger},
  title = {{Deep Interpretable Classification and Weakly-Supervised Segmentation of Histology Images via Max-Min Uncertainty}},
  journal = {IEEE Trans. Medical Imaging},
  year = {2022},
  doi = {10.1109/tmi.2021.3123461},
}
Figure