Democratizing Pathological Image Segmentation with Lay Annotators via Molecular-Empowered Learning

Ruining Deng, Yanwei Li, Peize Li, Jiacheng Wang, Lucas W. Remedios, Saydolimkhon Agzamkhodjaev, Zuhayr Asad, Quan Liu, Can Cui, Yaohong Wang, Yihan Wang, Yucheng Tang, Haichun Yang, Yuankai Huo

TL;DR

This paper introduces a novel "molecular-empowered learning" framework that enables lay annotators, without medical expertise, to achieve high-quality cell segmentation in pathology images. By integrating molecular imaging with anatomical images and employing a corrective learning strategy, the method significantly outperforms traditional annotation by experienced pathologists, democratizing AI development in digital pathology.

What Problem Does This Paper Address?

Accurate cell segmentation in digital pathology is crucial for diagnosing diseases and understanding their progression. However, training AI models for this task typically requires extensive, pixel-level manual annotations from highly trained experts like pathologists. This process is not only time-consuming and expensive but also prone to errors, especially when distinguishing subtle cell types. The paper aims to overcome this bottleneck by exploring whether non-experts (lay annotators) can be empowered to perform accurate annotations, thereby democratizing the development of AI models for pathology.

Methods

The core of the proposed solution is a "molecular-empowered learning scheme" with three key components:

  1. Morphology-Molecular Multi-Modality Registration: This step aligns high-resolution anatomical images (PAS staining) with molecular images (immunofluorescence, IF) at both the whole slide image (WSI) and regional levels. This ensures that anatomical features in PAS images correspond precisely to molecular signals in IF images, providing crucial context for annotation. They use a two-stage registration process involving Map3D for global alignment and AIRLab for precise pixel-level correspondence.
  2. Molecular-Informed Lay Annotation: With the registered images, lay annotators are guided by the molecular signals to identify and label different cell types (e.g., podocytes and mesangial cells). This approach leverages the distinct fluorescent signals of biomolecules to differentiate cells that might appear similar morphologically. They use ImageJ with synchronized windows and a "ROI Manager" for efficient annotation.
  3. Molecular-Oriented Corrective Learning: To address the inherent noise and potential errors in lay annotations, a deep corrective learning strategy is employed. This method identifies regions where the AI model and the lay annotator strongly agree, using these high-confidence areas to refine the learning process. It leverages top-k pixel feature embeddings and cosine similarity to highlight consensus regions, effectively mitigating the impact of noisy labels.

Datasets

The study utilized 11 PAS staining WSIs, including 3 injured glomeruli slides, paired with corresponding IF images. After registration and segmentation, 1,147 patches for podocyte cells and 789 patches for mesangial cells were generated, each with a resolution of 512x512 pixels. The paper also mentions using a "gold standard" dataset created by experienced pathologists for comparison.

Key Insights and Ideas

  • Molecular Signals as Annotation Guides: The most significant insight is that molecular imaging (IF) can act as a powerful guide for non-experts to accurately differentiate cell types that are morphologically ambiguous even to experienced pathologists. This bypasses the need for extensive domain knowledge.
  • Democratizing Annotation: The framework successfully demonstrates that lay annotators can achieve performance comparable to, and in some metrics, superior to experienced pathologists, drastically reducing the barrier to entry for pathology AI development.
  • Corrective Learning for Noisy Labels: The proposed corrective learning strategy is crucial for making lay annotations viable. It effectively filters out noise and amplifies agreement between the model and annotators, leading to more robust training.
  • Efficiency Gains: By enabling lay annotators, the potential for scaling up annotation efforts is immense, making the development of large-scale pathology AI models more feasible and cost-effective.

Key Results

  • Superior Annotation Accuracy: Molecular-informed annotations from lay annotators achieved an average F1 score of 0.8496, significantly outperforming morphology-based annotations from experienced pathologists (0.7015).
  • Higher Inter-Rater Agreement: The Fleiss' kappa test indicated higher annotation agreement among lay annotators using the molecular-informed approach compared to experts using morphology alone.
  • Improved Segmentation Performance: The proposed partial label segmentation method, leveraging the molecular-empowered learning, showed superior performance on multi-class cell segmentation compared to several baseline models.
  • Efficiency: While not explicitly quantified in terms of time saved per annotation, the paper implies significant efficiency gains by reducing the need for highly specialized and time-consuming expert annotations.

Limitations and Future Work

  • Annotation Variability: While the molecular-informed approach improves accuracy, the paper acknowledges that lay annotations can still be noisy and error-prone, necessitating the corrective learning strategy.
  • Generalizability: The study focuses on specific cell types (podocytes and mesangial cells) in renal pathology. Future work could explore the applicability of this framework to a wider range of cell types and tissue types.
  • Computational Cost: Multi-modality registration and deep learning models can be computationally intensive. Optimizing these processes for broader accessibility could be a future direction.
  • Further Refinement of Corrective Learning: The paper presents an ablation study on corrective learning designs, suggesting that further exploration into optimizing these strategies could yield even better results.

Summary updated

Abstract

Multi-class cell segmentation in high-resolution Giga-pixel whole slide images (WSI) is critical for various clinical applications. Training such an AI model typically requires labor-intensive pixel-wise manual annotation from experienced domain experts (e.g., pathologists). Moreover, such annotation is error-prone when differentiating fine-grained cell types (e.g., podocyte and mesangial cells) via the naked human eye. In this study, we assess the feasibility of democratizing pathological AI deployment by only using lay annotators (annotators without medical domain knowledge). The contribution of this paper is threefold: (1) We proposed a molecular-empowered learning scheme for multi-class cell segmentation using partial labels from lay annotators; (2) The proposed method integrated Giga-pixel level molecular-morphology cross-modality registration, molecular-informed annotation, and molecular-oriented segmentation model, so as to achieve significantly superior performance via 3 lay annotators as compared with 2 experienced pathologists; (3) A deep corrective learning (learning with imperfect labels) method is proposed to further improve the segmentation performance using partially annotated noisy data. From the experimental results, our learning method achieved F1 = 0.8496 using molecular-informed annotations from lay annotators, which is better than conventional morphology-based annotations (F1 = 0.7015) from experienced pathologists. Our method democratizes the development of a pathological segmentation deep model to the lay annotator level, which consequently scales up the learning process similar to a non-medical computer vision task. The official implementation and cell annotations are publicly available at https://github.com/hrlblab/MolecularEL .
BibTeX
@inproceedings{deng2023democratizing,
  author = {Ruining Deng and Yanwei Li and Peize Li and Jiacheng Wang and Lucas W. Remedios and Saydolimkhon Agzamkhodjaev and Zuhayr Asad and Quan Liu and Can Cui and Yaohong Wang and Yihan Wang and Yucheng Tang and Haichun Yang and Yuankai Huo},
  title = {{Democratizing Pathological Image Segmentation with Lay Annotators via Molecular-Empowered Learning}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2023},
  url = {https://openreview.net/forum?id=BEFuUUl_gEp},
}
Figure