TL;DR
This paper introduces "PathProfiler," an AI tool designed to automatically assess the quality of digital histopathology slides (whole-slide images or WSIs) in a real-time clinical setting. By flagging potentially suboptimal images, PathProfiler aims to improve diagnostic accuracy and efficiency, reducing the burden of manual quality control.
What Problem Does This Paper Address?
The widespread adoption of digital pathology (DP) and the integration of AI tools for diagnosis are hampered by the critical need for high-quality digitized whole-slide images (WSIs). Currently, WSI quality control relies heavily on manual assessment, which is time-consuming, subjective, and prone to inter-observer variability. This inefficiency can lead to delays in diagnosis and potentially impact the reliability of AI algorithms that depend on image quality. The paper addresses the challenge of automating this quality control process within a busy clinical workflow.
Methods
The study leverages "PathProfiler," a previously developed open-source AI tool for automated WSI quality assessment. PathProfiler works by analyzing image patches to predict quality measures like focus and staining, and then aggregates these into a slide-level "Usability Score" (US). For this study, the tool was integrated into a real-time clinical workflow. WSIs with a US below a threshold of 0.4 were flagged for manual review by Biomedical Scientists (BMS) and pathologists. The study also explored the utility of "heatmaps" generated by PathProfiler to visually indicate areas of concern.
Datasets
The study analyzed a total of 1254 genitourinary (GU) WSIs from 184 GU cases. This dataset included various specimen types, with 46% (574 WSIs) being prostate biopsies. PathProfiler was initially trained on prostate tissue, and its performance was evaluated on this dataset and other GU specimen types. The study also used historical data from comparable 3-week periods before and after the study to assess the impact on turnaround time.
Key Insights and Ideas
- AI for QC is Feasible in Clinical Settings: The study demonstrates that an AI-based quality control tool like PathProfiler can be successfully integrated into a real-time clinical pathology workflow, offering a more objective and efficient alternative to manual checks.
- Focus is a Major Culprit: Similar to existing literature, focus-related issues were the primary reason for WSIs being flagged as suboptimal by PathProfiler.
- Prostate Biopsies are a Strong Use Case: PathProfiler showed particularly good performance and concordance with subjective assessment for prostate biopsy cases, likely due to its initial training on this tissue type.
- Resection Cases Pose a Challenge: The tool was less reliable in assessing non-prostate surgical resection cases, highlighting the need for further training or adaptation for diverse specimen types.
- Potential for Workflow Improvement: While not significantly impacting overall turnaround time in this short study, the tool offers the potential to streamline QC, reduce pathologist frustration, and improve the overall acceptance of digital pathology.
Key Results
- Out of 1254 analyzed WSIs, 83 (6.6%) were flagged by PathProfiler as having a Usability Score (US) of 0.4 or less.
- For prostate biopsy WSIs (46% of the total), 4.5% (26/574) were flagged.
- Of the flagged WSIs, 54% were considered severe enough to warrant remedial action, with focus issues being the main concern.
- Remedial actions (cleaning and re-scanning) led to improved US scores and subjective quality in a significant portion of flagged WSIs, particularly for prostate biopsies (100% improvement in US after re-scans).
- PathProfiler's performance showed better concordance with subjective assessment for non-resection cases (especially prostate biopsies) compared to resection cases.
- The study did not find a statistically significant negative impact on diagnostic turnaround time (TAT) for prostate biopsy cases during the 3-week integration period.
Limitations and Future Work
- False-Negative Rate Not Assessed: The study did not explicitly assess the rate at which PathProfiler might miss suboptimal WSIs that pass the QC stage. This would require pathologists to actively flag images that were missed by the AI.
- Performance on Resection Cases: PathProfiler's reliability was lower for non-prostate surgical resection cases, suggesting a need for further development or training on a broader range of specimen types.
- Impact of US Threshold: The authors suggest that adjusting the US cut-off threshold (currently 0.4) could potentially improve specificity for genuinely severely impacted WSIs, reducing the burden of manual review for less critical issues.
- Future Work: The authors plan to investigate PathProfiler's utility across a wider range of specimen types and explore routine adoption. They also emphasize the growing importance of quality WSI for the development and reliability of AI tools in pathology.
Summary updated