TL;DR
This paper introduces a novel, large-scale, multi-species dataset of pulmonary hemosiderophages, crucial for diagnosing lung bleeding across equines, humans, and felines. They developed an expert-guided deep learning pipeline to create this dataset, enabling more robust inter-species cell detection models and advancing research in comparative pathology.
What Problem Does This Paper Address?
Diagnosing pulmonary hemorrhage (P-Hem), a condition involving bleeding into the lungs, often relies on analyzing bronchoalveolar lavage fluid (BALF) for hemosiderin-laden macrophages (hemosiderophages). While a standardized scoring system exists, accurately quantifying these cells, especially across different species, is challenging. Existing datasets are often species-specific, limited in size, or lack comprehensive annotations, hindering the development of generalized deep learning models for P-Hem detection. This scarcity of high-quality, multi-species data limits comparative pathology research and the potential for transfer learning between species, which could be invaluable for understanding human diseases through animal models.
Methods
The authors developed a sophisticated annotation pipeline that combines human expertise with deep learning and data visualization techniques. They started with a pre-trained deep learning object detection model (RetinaNet) initially trained on equine whole slide images (WSIs). This model was then used to perform "inference" on unannotated WSIs from equine, feline, and human samples, generating a large number of candidate cell detections.
To refine these initial detections, a multi-step semi-automatic cleaning process was employed:
- Clustering and Semi-automatic Data Cleaning: Cells were clustered based on their size and shape. This allowed for the efficient identification and removal of obvious false positives (e.g., small cell types, non-maximum suppression artifacts) by a human expert using a specialized web interface. This step also helped in re-labeling misclassified cells, particularly granulocytes in feline samples due to their smaller size.
- Expert Screening: A trained pathologist meticulously reviewed all algorithmic predictions across the WSIs. This involved a patch-by-patch screening process to correct errors and ensure dataset consistency.
- Density Map Visualization: To address the challenge of subjective grading of hemosiderin content, a regression-based approach was used to assign a continuous grade to each cell. These grades were then visualized on "density maps," where cells were arranged by their hemosiderin score. This visualization aided the expert in identifying and correcting misclassifications, especially at the boundaries between grades.
Datasets
The paper introduces a novel, fully annotated multi-species dataset comprising 74 cytology whole slide images (WSIs):
- Equine: 55 samples (16 for initial manual annotation, 39 for expert-algorithm labeling).
- Feline: 7 samples.
- Human: 12 samples.
This dataset contains a total of 297,383 manually verified hemosiderophages, classified into five grades based on their hemosiderin content. This dataset is highlighted as one of the largest publicly available WSI datasets in terms of the number of annotations, scanned area, and species covered for cytologic research questions.
Key Insights and Ideas
- The Power of Expert-Algorithm Collaboration: The study demonstrates that combining the speed and scale of deep learning with the precision and nuance of human expertise is highly effective for creating large, high-quality annotated datasets, especially for complex biological images.
- Bridging the Species Gap: The creation of a multi-species dataset is a significant step towards developing generalized deep learning models that can perform cell detection across different species. This has broad implications for comparative pathology and the use of animal models in human disease research.
- Addressing Annotation Subjectivity: The development and use of density maps to visualize and refine cell grading is a novel approach to tackle the inherent inter- and intra-observer variability in subjective scoring systems.
- Transfer Learning Potential: The findings suggest that deep learning models trained on one species (e.g., equine) can be effectively adapted to other species with minimal retraining, opening doors for efficient model development in data-scarce domains.
Key Results
- Dataset Size: A total of 297,383 hemosiderophages were annotated across 74 WSIs from equine, feline, and human samples.
- Inter-species Transfer Learning: A deep learning model trained on equine data achieved a mean average precision (mAP) of 0.88 on human data and 0.79 on feline data, indicating good transferability without extensive retraining.
- Ablation Study: The study found that performance significantly improved with increasing numbers of training patches, converging around 128 patches (approximately 1000 unique hemosiderophages) from a single WSI. Further increases in data did not yield substantial gains.
- Annotation Refinement: The expert screening process led to the deletion of a substantial portion of initially detected cells (44.8% in screening, 17.45% in semi-automatic cleaning), highlighting the importance of rigorous quality control.
Limitations and Future Work
- Limitations:
- The initial deep learning model, while sensitive, had low specificity, leading to a large number of false positives that required extensive manual correction.
- The staining process, particularly for iron, can introduce difficulties in differentiating cell types, impacting annotation accuracy.
- The feline dataset, while valuable, is smaller than the equine and human datasets, which might limit the robustness of models trained solely on feline data.
- Future Work:
- The authors suggest that further studies are needed to explore the application of these algorithms for specific human diseases, such as COVID-19.
- Investigating the transferability of these models to other species, such as dogs, is also proposed.
- The paper highlights the potential for developing more efficient methods to reduce manual interactions by focusing on optimal threshold selection for object detection models.
Summary updated