TL;DR
This paper introduces TAPAS, a novel automated algorithm for segmenting white matter lesions in Multiple Sclerosis (MS) brain MRI scans. TAPAS addresses a critical limitation in existing methods by automatically determining a subject-specific threshold for probability maps, thereby reducing human bias and improving the accuracy of lesion volume estimation.
What Problem Does This Paper Address?
Estimating the volume of white matter lesions (WMLs) in the brain is crucial for tracking disease progression and treatment efficacy in Multiple Sclerosis (MS). While automated segmentation methods exist, they often produce "probability maps" that require a threshold to create binary lesion masks. The problem is that this threshold is frequently chosen manually or using a single group-level value, introducing human error, bias, and potentially leading to inaccurate volume estimates, especially for individuals with very small or very large lesion loads. This paper aims to solve this by developing a method that automatically determines an optimal, subject-specific threshold.
Methods
The core of the TAPAS method involves two main steps:
- Finding the "true" subject-specific threshold: For each subject, TAPAS uses an existing automatic segmentation algorithm (MIMOSA in this study) to generate a probability map. It then systematically tests a range of thresholds (from 0% to 100% in 1% increments) on this map. For each threshold, it calculates the Sørensen-Dice Similarity Coefficient (DSC) against the manually segmented lesion mask. The threshold that yields the maximum DSC for that specific subject is identified as the "true" subject-specific threshold.
- Modeling subject-specific thresholds: To predict this optimal threshold for new subjects without manual segmentation, TAPAS models the relationship between a naive volume estimate (obtained using a group-level threshold) and the "true" subject-specific threshold. This is done using a generalized additive model (GAM). The GAM allows for flexible, non-linear relationships, and the model is trained on a portion of the data. Once trained, the model can predict a subject-specific threshold based on the naive volume estimate derived from their probability map.
Datasets
The study utilized two datasets:
- Johns Hopkins Hospital (JHH) dataset: 98 subjects with MS, scanned on a Philips 3T MRI scanner. (4 subjects were excluded due to poor image quality, leaving 94).
- Brigham and Women's Hospital (BWH) dataset: 40 subjects with MS, scanned on a Siemens 3T MRI scanner.
The data was processed using a Monte Carlo-resampled split-sample cross-validation approach, where 100 different train/test splits were generated to ensure robust evaluation.
Key Insights and Ideas
- Subject-specificity matters: The most significant insight is that a single, group-level threshold is insufficient for accurate lesion segmentation across all individuals. Subject-specific thresholds are crucial for capturing the variability in lesion appearance and size.
- Bias mitigation: TAPAS effectively mitigates volumetric bias that can arise from group-level thresholding, particularly in subjects with extreme lesion loads. This is a critical improvement for clinical studies where accurate volume is paramount.
- Generalizability: The method is designed to be independent of the specific preprocessing pipeline used, making it adaptable to data from different scanners and protocols.
- Transparency and speed: TAPAS is presented as a fully transparent and fast algorithm, making it practical for widespread adoption.
Key Results
- Reduced Absolute Error: In the JHH dataset, TAPAS achieved an average reduction in subject-level absolute error of 0.1 mL per 1 mL increase in manual volume. The average absolute error (AE) for TAPAS was 2.21 mL in JHH data, compared to 2.7 mL for group thresholding.
- Improved DSC: The average Sørensen-Dice Coefficient (DSC) for TAPAS was 0.61 in JHH data, slightly higher than the 0.60 for group thresholding. In BWH data, TAPAS achieved an average DSC of 0.66, matching group thresholding.
- Bias Reduction: Bland-Altman plots showed that TAPAS significantly reduced systematic bias observed with group-level thresholding in the JHH data, where lesions were often underestimated in subjects with larger lesion loads. In the BWH data, where initial bias was minimal, TAPAS maintained unbiased estimation.
- Phenotypic Associations: Correlation analyses showed that lesion volumes derived from TAPAS and group thresholding had similar associations with clinical variables (like EDSS score and disease duration) as manual segmentations, supporting the clinical relevance of the method.
Limitations and Future Work
- Dependency on continuous maps: TAPAS requires probability maps generated by an initial automatic segmentation algorithm. It does not perform the initial segmentation itself.
- Model fit variability: The performance of the TAPAS model might be affected by small training datasets, uniform lesion loads, or significant differences between training and testing data.
- Longitudinal data: The current study focused on cross-sectional data. Future work could extend TAPAS to analyze longitudinal lesion volumetrics.
- Other segmentation approaches: TAPAS was validated with one specific segmentation algorithm (MIMOSA). Future research should explore its performance with other automatic segmentation methods, as their probability map characteristics might differ.
- Dynamic thresholds: The authors suggest that for very small or very large lesions, even subject-specific thresholds might not be optimal, and dynamic thresholds within a subject could be a future direction.
Summary updated