Abstract
Accurate lesion segmentation is critical in stroke rehabilitation research for the quantification of lesion burden and accurate image processing. Current automated lesion segmentation methods for T1-weighted (T1w) MRIs, commonly used in rehabilitation research, lack accuracy and reliability. Manual segmentation remains the gold standard, but it is time-consuming, subjective, and requires significant neuroanatomical expertise. We previously released a large, open-source dataset of stroke T1w MRIs and manually segmented lesion masks (ATLAS v1.2, N=304) to encourage the development of better algorithms. However, many methods developed with ATLAS v1.2 report low accuracy, are not publicly accessible or are improperly validated, limiting their utility to the field. Here we present ATLAS v2.0 (N=955), a larger dataset of T1w stroke MRIs and manually segmented lesion masks that includes both training (public) and test (hidden) data. Algorithm development using this larger sample should lead to more robust solutions, and the hidden test data allows for unbiased performance evaluation via segmentation challenges. We anticipate that ATLAS v2.0 will lead to improved algorithms, facilitating large-scale stroke rehabilitation research.
Introduction: The Challenge of Automated Segmentation
In stroke rehabilitation research, identifying the precise location and volume of a lesion (lesion burden) is essential for predicting patient outcomes. While manual segmentation by experts is the current "gold standard," it is labor-intensive and requires specialized neuroanatomical knowledge. Automated algorithms offer a potential solution, but they frequently struggle with T1-weighted (T1w) MRIs—the high-resolution standard in rehabilitation—due to the lack of contrast found in acute-phase multispectral imaging.
Previous efforts to automate this process were hindered by small datasets and a lack of "hidden" test sets, which often led to model overfitting. Consequently, many existing tools fail to generalize to new, real-world scans.
The ATLAS v2.0 Dataset
To overcome these hurdles, we developed ATLAS v2.0 (Anatomical Tracings of Lesions After Stroke), a curated dataset of 955 T1w MRIs and manually segmented lesion masks. This release triples the size of the previous version and standardizes data from 33 research cohorts across 20 institutions worldwide.
Rigorous Standardization and Quality Control
To ensure the dataset is robust for machine learning, we employed a strict manual pipeline:
- Expert Tracing: Trained tracers used ITK-SNAP to delineate lesions in native space.
- Quality Control: Every mask underwent review and editing by at least two additional researchers to ensure boundary accuracy.
- Preprocessing: The team released both raw data and a preprocessed version featuring intensity normalization, registration to the MNI-152 template, and "defacing" to protect participant privacy.
- BIDS Compliance: Data is organized according to the Brain Imaging Data Structure (BIDS), ensuring compatibility with standard neuroimaging software.
Dataset Composition
The dataset is structured to facilitate both training and unbiased evaluation:
- Public Training Set (n=655): Includes T1w MRIs and corresponding manual lesion masks.
- Hidden Test Set (n=300): Only T1w scans are public; masks are withheld for fair performance evaluation in challenges.
- Generalizability Set (n=135): A completely hidden set from entirely new cohorts used to test how well algorithms handle unseen data.
Key Insights: Bridging the Reliability Gap
A major finding in current research is the "reliability gap." While clinical utility typically requires a Dice coefficient (a measure of spatial overlap) of at least 0.85, most existing public methods only achieve scores between 0.5 and 0.7.
As shown in the table above, there is significant variability in how current methods are validated and the performance they report. ATLAS v2.0 addresses this by introducing hidden test sets to enforce scientific honesty and ensure models generalize across diverse stroke presentations, including unilateral, bilateral, and brainstem lesions.
Conclusion and Future Work
ATLAS v2.0 is now the largest open-source dataset of its kind. By providing a massive, curated training set alongside hidden test data, we aim to bridge the gap between current automated performance and the reliability needed for clinical research. While currently focused on T1w MRIs, future work will involve using this data to fuel international segmentation challenges, encouraging the development of 3D convolutional neural networks (CNNs) that can finally match human performance in the clinic.
Summary updated