Deep Learning with Permutation-invariant Operator for Multi-instance Histopathology Classification

Jakub M. Tomczak, Maximilian Ilse, Max WellingarXiv 2017 · arXiv

Introduction

Digital pathology faces a significant "small $n$, large $p$" challenge. While biopsy slides contain millions of pixels, researchers often only have a single diagnosis per patient. This weak supervision makes training standard deep learning models difficult, as pixel-level annotations are rarely available. This paper proposes a Deep Multi-Instance Learning (MIL) framework to address these bottlenecks by treating slides as "bags" of small patches.

Methodology

The core of the proposed framework is the use of permutation-invariant operators. In the MIL paradigm, a slide is considered a "bag" and its constituent patches are "instances." A bag is labeled positive if at least one instance within it is positive.

To aggregate patch-level scores into a slide-level diagnosis, the authors evaluated three differentiable operators:

  1. Noisy-Or (NOR): A probabilistic logic operator where a bag is negative only if all patches are negative.
  2. Integrated Segmentation and Recognition (ISR): A ratio-based aggregator.
  3. Log-Sum-Exp (LSE): A smooth approximation of the maximum function.

Unlike traditional multi-stage MIL methods, this approach allows for end-to-end training via backpropagation, enabling the model to learn discriminative features directly from the data.

Datasets and Preprocessing

The study utilized 58 H&E stained histopathology images from 32 benign and 26 malignant breast cancer patients. Given the small dataset size, extensive data augmentation was employed to ensure model robustness.

Data augmentation techniques including rotation, blurring, and H&E normalization applied to histopathology patches.

As shown in the figure above, the preprocessing pipeline included H&E color decomposition to adjust stain intensity, random rotations, mirroring, and Gaussian blurring. These techniques help the model generalize better by simulating variations commonly found in clinical settings.

Results and Discussion

The Deep MIL models were compared against state-of-the-art Gaussian Process-based MIL (GPMIL).

Method Accuracy AUC
DEEPNOR-MIL (Noisy-Or) 0.879 0.88
DEEPISR-MIL 0.828 0.90
RGPMIL (State-of-the-art) N/A 0.90

The results indicate that Deep MIL achieves performance comparable to state-of-the-art methods. Specifically, the Noisy-Or operator provided the most balanced performance in terms of accuracy and F-score. Furthermore, the model implicitly learns to identify Regions of Interest (ROIs) by assigning higher scores to patches that contribute most to the final diagnosis, effectively creating diagnostic heatmaps.

Conclusion

This work demonstrates that permutation-invariant operators in a Deep MIL framework can effectively classify histopathology slides under weak supervision. While the small dataset remains a limitation, the ability to train end-to-end and identify ROIs without pixel-level labels represents a significant step forward in automated cancer diagnosis. Future work may explore Bayesian deep learning to improve uncertainty quantification in clinical applications.

Summary updated

Abstract

The computer-aided analysis of medical scans is a longstanding goal in the medical imaging field. Currently, deep learning has became a dominant methodology for supporting pathologists and radiologist. Deep learning algorithms have been successfully applied to digital pathology and radiology, nevertheless, there are still practical issues that prevent these tools to be widely used in practice. The main obstacles are low number of available cases and large size of images (a.k.a. the small n, large p problem in machine learning), and a very limited access to annotation at a pixel level that can lead to severe overfitting and large computational requirements. We propose to handle these issues by introducing a framework that processes a medical image as a collection of small patches using a single, shared neural network. The final diagnosis is provided by combining scores of individual patches using a permutation-invariant operator (combination). In machine learning community such approach is called a multi-instance learning (MIL).
BibTeX
@misc{tomczak2017deep,
  author = {Jakub M. Tomczak and Maximilian Ilse and Max Welling},
  title = {{Deep Learning with Permutation-invariant Operator for Multi-instance Histopathology Classification}},
  year = {2017},
  eprint = {1712.00310},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1712.00310},
}
Figure