Weakly-Supervised Semantic Segmentation for Histopathology Images Based on Dataset Synthesis and Feature Consistency Constraint

Zijie Fang, Yang Chen, Yifeng Wang, Zhi Wang, Xiangyang Ji, Yongbing Zhang

Introduction

In computational pathology, segmenting various tissue types, such as tumors versus stroma, is a critical step for accurate patient prognosis. However, training deep learning models for semantic segmentation typically necessitates pixel-level annotations. This process is notoriously labor-intensive, time-consuming, and requires the expertise of trained pathologists.

To mitigate these challenges, researchers have turned to Weakly-Supervised Semantic Segmentation (WSSS), which utilizes simpler image-level labels (e.g., "tumor present"). Most existing WSSS frameworks rely on Class Activation Maps (CAMs) to estimate tissue locations. However, CAMs are primarily designed for classification and tend to highlight only the most discriminative regions, resulting in imprecise boundaries and incomplete masks. Histopathology images, characterized by high complexity and repetitive patterns, further exacerbate these limitations in standard CAM-based methods.

Methodology: The PistoSeg Framework

The authors propose PistoSeg, a novel framework that shifts the paradigm from pure CAM-based estimation to a synthesis and refinement approach. The workflow is structured into three primary stages:

1. Synthesized Dataset Generation

The authors observed that many pathology slides contain regions of a single tissue type. By employing a "Mosaic Transformation," they "cut and paste" these single-tissue samples into a grid to create synthetic multi-tissue images. This approach provides "free" pixel-level ground truth masks for the synthetic data, as the exact location of each tissue component is known.

2. Preliminary Segmentation and Pseudo-Mask Refining

A preliminary segmentation model (Unet++) is trained on this synthetic dataset. While this model learns to distinguish tissue types, it often struggles with the artificial boundaries of synthetic images. To bridge this gap, PistoSeg introduces a Pseudo-Mask Refining Module utilizing an attention-based feature consistency constraint. This module processes real images through two parallel paths: the preliminary model (generating logits) and a CAM-based model. By enforcing feature consistency in an attention space between these paths, the framework produces significantly sharper and more accurate "pseudo-masks" for real training data.

The PistoSeg framework overview, illustrating the three-stage process: synthesized dataset generation, pseudo-mask refinement, and final precise segmentation.

3. Final Precise Segmentation

The refined pseudo-masks serve as the ground truth for training a final, high-precision segmentation model, effectively transferring the knowledge gained from synthetic data to real-world histopathology images.

Experimental Setup

PistoSeg was evaluated on two major benchmarks:

  • WSSS4LUAD: A Lung Adenocarcinoma dataset containing over 10,000 training images (tumor, stroma, and normal tissues).
  • BCSS-WSSS: A Breast Cancer Semantic Segmentation dataset with over 23,000 images, covering tumor, stroma, lymphocytic infiltrate, and necrosis.

Results and Discussion

PistoSeg demonstrates state-of-the-art performance across both benchmarks. On the WSSS4LUAD dataset, it achieved a mean Intersection over Union (mIoU) of 0.7513, surpassing the previous leading method, OEEM (0.7054). Notably, on the BCSS-WSSS dataset, PistoSeg (mIoU 0.7055) outperformed fully-supervised models in specific categories like lymphocytic infiltrate (LYM) and necrosis (NEC), likely due to the data balancing inherent in the synthesis module.

Qualitative analysis confirms that PistoSeg produces masks with cleaner boundaries that closely align with actual tissue morphology, whereas competing methods often suffer from boundary "bleeding" or incomplete coverage.

Qualitative comparison of segmentation results. PistoSeg (ours) demonstrates superior boundary adherence and mask completeness compared to other weakly-supervised methods.

Conclusion

PistoSeg effectively addresses the limitations of CAM-based WSSS in pathology by leveraging the repetitive nature of tissue through dataset synthesis and a novel feature consistency constraint. While the current Mosaic transformation uses simple rectangular boundaries, future work may explore generative models to create even more realistic synthetic training data. Despite its reliance on initial single-tissue samples, PistoSeg represents a significant advancement toward reducing the annotation burden in computational pathology.

Summary updated

Abstract

Tissue segmentation is a critical task in computational pathology due to its desirable ability to indicate the prognosis of cancer patients. Currently, numerous studies attempt to use image-level labels to achieve pixel-level segmentation to reduce the need for fine annotations. However, most of these methods are based on class activation map, which suffers from inaccurate segmentation boundaries. To address this problem, we propose a novel weakly-supervised tissue segmentation framework named PistoSeg, which is implemented under a fully-supervised manner by transferring tissue category labels to pixel-level masks. Firstly, a dataset synthesis method is proposed based on Mosaic transformation to generate synthesized images with pixel-level masks. Next, considering the difference between synthesized and real images, this paper devises an attention-based feature consistency, which directs the training process of a proposed pseudo-mask refining module. Finally, the refined pseudo-masks are used to train a precise segmentation model for testing. Experiments based on WSSS4LUAD and BCSS-WSSS validate that PistoSeg outperforms the state-of-the-art methods. The code is released at https://github.com/Vison307/PistoSeg.
BibTeX
@inproceedings{fang2023weakly,
  author = {Zijie Fang and Yang Chen and Yifeng Wang and Zhi Wang and Xiangyang Ji and Yongbing Zhang},
  title = {{Weakly-Supervised Semantic Segmentation for Histopathology Images Based on Dataset Synthesis and Feature Consistency Constraint}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i1.25136},
}
Figure