TL;DR
This paper introduces BrainSec, an automated pipeline for segmenting grey matter (GM) and white matter (WM) in gigapixel whole-slide images (WSIs) of brain tissue. This is crucial for scalable neuropathological analysis, particularly for diseases like Alzheimer's, by enabling automated quantification and visualization of pathologies within specific brain regions.
What Problem Does This Paper Address?
Neurodegenerative diseases like Alzheimer's are characterized by pathological hallmarks that can appear in both grey and white matter. Traditionally, identifying and quantifying these hallmarks involves manual inspection of brain tissue slides by experts. This process is time-consuming, subjective, and not scalable for analyzing large datasets of high-resolution whole-slide images (WSIs). Existing automated methods often struggle with the sheer size of WSIs, the variability in staining, and the presence of artifacts. The paper aims to address this by developing an automated, robust, and scalable pipeline for segmenting GM and WM regions, which is a foundational step for subsequent pathological analysis.
Methods
The core of the proposed solution is the BrainSec pipeline, which consists of two main modules:
- CNN Module: This module transforms the pixel-level segmentation problem into a patch-based classification problem.
- Patch-Based Classification: It utilizes a modified ResNet-18 architecture, pre-trained on ImageNet, to extract features from 256x256 pixel patches. The final layer is adapted to classify patches into three categories: GM, WM, or background. A weighted cross-entropy loss function is used to handle class imbalance, with weights inversely proportional to class frequencies.
- Pixel-Wise Segmentation: The patch classifications are then used to construct pixel-wise segmentation masks. This is achieved through a sliding window approach, where the step size determines the resolution of the output masks. A smaller step size leads to higher accuracy but increased computational complexity.
- Post-Processing Module: This module refines the segmentation masks by removing artifacts and unwanted tissue fragments. It involves downsampling, applying area opening and closing operations, and morphological opening to smooth boundaries.
Additionally, the authors propose BrainSec+, which integrates a Neural Conditional Random Field (NCRF) layer to model spatial relationships between neighboring patches. This aims to alleviate the need for a separate post-processing module and potentially improve robustness.
Datasets
The study utilizes several datasets:
- 30-WSI Dataset: This dataset, containing 30 WSIs from formalin-fixed paraffin-embedded temporal cortex sections stained for Amyloid-β, was used for initial testing, training, and validation. It includes 18 Alzheimer's disease (AD) cases and 12 non-Alzheimer's disease (NAD) cases.
- 130-WSI Dataset: This larger dataset comprises 130 temporal WSIs from new cases, used to rigorously test the model's performance on a broader cohort. It contains 117 AD cases and 13 NAD cases.
- 52-Heteroarea Dataset: This dataset includes 52 WSIs from frontal and parietal lobes, areas not seen during training, to evaluate the model's generalization to different neuroanatomical regions. It contains 26 cases, with some overlap with the other datasets.
For training the patch-based classification, approximately 310k patches from GM, 100k from WM, and 300k from background were generated.
Key Insights and Ideas
- Patch-based approach for gigapixel WSIs: Directly segmenting gigapixel WSIs is computationally prohibitive. Transforming the problem into a patch-based classification task with a robust CNN backbone like ResNet-18 offers a practical solution.
- Flexibility in step size: The ability to adjust the sliding window step size allows for a trade-off between segmentation accuracy and computational efficiency, making the pipeline adaptable to different needs.
- Importance of post-processing: Artifacts in WSIs are a significant challenge. The proposed post-processing module effectively cleans up segmentation masks, improving their reliability.
- BrainSec+ for automation: Integrating NCRF into BrainSec+ demonstrates a promising direction for fully automated segmentation without manual post-processing tuning, though it comes with increased computational cost.
- Feature extractor robustness: Grad-CAM analysis suggests that BrainSec's feature extractor relies on differential tissue textures rather than pathological hallmarks, indicating robustness across AD and NAD cases.
Key Results
- Quantitative Performance: BrainSec and BrainSec+ consistently outperformed FCN and U-Net across various metrics (IoU, Dice Coefficient, F1-score) on hold-out test sets. For example, BrainSec achieved a mean IoU of 96.64 ± 1.51 for AD GM, significantly higher than FCN (75.46 ± 11.0) and U-Net (97.32 ± 0.61). BrainSec+ further improved performance, particularly in WM segmentation for NAD cases.
- Robustness: BrainSec/BrainSec+ showed lower standard deviations in IoU scores, indicating more consistent and robust performance across heterogeneous slides and different disease statuses (AD vs. NAD).
- Generalization: The model demonstrated stable performance on broader neuroanatomical areas (frontal and parietal lobes) and temporal WSIs, achieving accurate segmentation in over 94% of the 130-WSI dataset.
- Integration Capability: BrainSec was successfully integrated into an existing plaque identification model, enabling automated quantification and visualization of Amyloid-β plaques in GM/WM regions.
- Computational Efficiency: BrainSec/BrainSec+ required significantly less GPU memory (around 700 MB) compared to FCN and U-Net (11.9 GB), making it more suitable for edge deployments.
Limitations and Future Work
- Computational Power: The current study requires significant computing power and time to process each WSI, with a single gigapixel WSI taking about 6 hours on a NVIDIA Tesla T4 GPU. While deep learning methods offer generalizability, GPU requirements can be a limitation.
- Training Data Diversity: The current training set primarily consists of temporal regions. Future work aims to test BrainSec on more neuroanatomical areas and investigate performance improvements with more diverse training data.
- File Formats and Scanners: The authors plan to implement BrainSec on different WSI file formats and explore its performance with data from various scanners.
- BrainSec+ vs. BrainSec: While BrainSec+ offers full automation, it incurs higher computational costs with only trivial performance gains on their dataset. The authors suggest BrainSec+ might be advantageous in resource-rich environments or for deployment in heterogeneous settings requiring manual post-processing avoidance.
Summary updated