Abstract
Pathological examination is crucial to cancer diagnosis. Typically, pathologists draw conclusions based on observed cell and tissue structures on histology slides. Rapid developments in machine learning, especially deep learning, have established robust and accurate classifiers to assist in this process. Most machine learning systems rely heavily on annotated datasets to perform tasks such as classification and segmentation. This work investigates different granularities of annotations in histopathological datasets—including image-wise, bounding box, ellipse-wise, and pixel-wise—to verify their influence on deep learning models. Our experiments show that state-of-the-art classifiers perform significantly better when trained on pixel-wise annotation datasets, with average improvements in precision, recall, and F1-score of 7.87%, 8.83%, and 7.85%, respectively. Similarly, semantic segmentation algorithms achieve 8.33% better accuracy with pixel-wise annotations. These findings suggest that finer-grained annotations not only improve model performance but also help extract more accurate phenotypic information, potentially aiding pathologists in more precise diagnosis.
Introduction
Pathological examination is the "gold standard" for cancer diagnosis, but it is a grueling, manual process. As cancer rates rise globally, the workforce of pathologists is shrinking, leading to massive increases in diagnostic workloads. While deep learning (DL) offers a solution through automated image analysis, these models are only as good as the data used to train them.
The core challenge lies in annotation granularity. Labeling every single pixel of a tumor cell (pixel-wise) is incredibly time-consuming for a trained pathologist, whereas drawing a rough square around it (bounding box) is much faster. This study investigates whether the extra effort of fine-grained labeling results in a superior model or if simpler, coarser annotations are sufficient for clinical applications.
Methodology
Granularity Levels and Models
The researchers conducted a comparative study across two primary tasks: Classification (identifying cancer grade) and Semantic Segmentation (outlining cell boundaries). They compared three types of labels:
- Bounding Box (Bb): A simple rectangle around the cell.
- Ellipse-wise (el): An oval fitting the cell shape.
- Pixel-wise (pw): A precise mask covering every pixel of the cell nucleus.
To ensure a fair comparison, the team used a "downgrade" approach: starting with high-quality pixel-wise labels and using OpenCV to mathematically generate ellipses and bounding boxes, ensuring the underlying image data remained identical. They tested standard architectures including ResNet18, VGG16, and MobileNet for classification, and UNet, FPN, and LinkNet for segmentation.
Datasets
The study primarily used images of clear cell renal carcinoma (ccRCC) sourced from The Cancer Genome Atlas (TCGA). Six rigorously trained experts annotated 200 diagnostic regions (800x800 pixels), grading cells according to the WHO/ISUP standard (Grades 1 and 2).
Results and Discussion
Quantitative Performance Gains
The shift from bounding boxes to pixel-wise annotations yielded significant quantitative gains across all tested models:
- Classification: On average, pixel-wise labels improved precision by 7.87%, recall by 8.83%, and F1-score by 7.85%.
- Segmentation Accuracy: In the UNet model, pixel-wise annotations achieved 95.79% accuracy—6.2% higher than bounding box labels.
Precision and Interpretability
Finer annotations allow models to extract more accurate "phenotypic information"—the specific physical characteristics like color, texture, and morphology of a cell. Coarse boxes often include "noise" from background tissue that can confuse the model. Furthermore, pixel-wise training enhances interpretability; by providing precise outlines, the model can show a pathologist exactly which features led to a diagnosis, thereby increasing clinical trust.
Conclusion
While pixel-wise labels are superior, they are undeniably labor-intensive to produce. However, the performance gains and the ability to extract accurate phenotypic information suggest that finer-grained annotation is a worthwhile investment for developing reliable clinical tools. Future work will explore genotype-phenotype associations, linking these precise visual features to underlying genetic mutations, and the potential for automated pathological reporting.
Summary updated