Analysis of the MICCAI Brain Tumor Segmentation -- Metastases (BraTS-METS) 2025 Lighthouse Challenge: Brain Metastasis Segmentation on Pre- and Post-treatment MRI
Nazanin Maleki, Raisa Amiruddin, Ahmed W. Moawad, Nikolay Yordanov, Athanasios Gkampenis, Pascal Fehringer, Fabian Umeh, Crystal Chukwurah, Fatima Memon, Bojan Petrović, Justin Cramer, Mark Krycia, Elizabeth B. Shrickel, Ichiro Ikuta, Gerard Thompson, Lorenna Vidal, Vilma Kosović, Adam Goldman-Yassen, Virginia Hill, Tiffany Y. So, Sedra Abou Ali Mhana, Abdullah Alotaibi, Nathan Page, Prateek Bhatia, Melisa S. Guelen, Yasaman Sharifi, Marko Jakovljevic, Salma As Abosabie, Sara A. Abosabie, Mohanad Ghonim, Mohamed Ghonim, Amirreza Manteghinejad, Anastasia Janas, Kiril Krantchev, Maruf Adewole, Jake Albrecht, Udunna Anazodo, Sanjay Aneja, Anwar, Syed Muhammad, Timothy Bergquist, Veronica Chiang, Verena Chung, Gian Marco Conte, Farouk Dako, J. Mark Eddy, Ivan Ezhov, Khalili, Nastaran, Keyvan Farahani, Juan Eugenio Iglesias, Zhifan Jiang, Elaine Johanson, Anahita Fathi Kazerooni, Florian Kofler, Dominic LaBella, Koen Van Leemput, Hongwei Li, Marius George Linguraru, Xinyang Liu, Zeke Meier, Bjoern Menze, Harrison Moy, Klara Osenberg, Marie Piraud, Zachary Reitman, Russell Takeshi Shinohara, Chunhao Wang, Benedikt Wiestler, Walter F. Wiggins, Umber Shafique, Klara Willms, Arman Avesta, Khaled Bousabarah, Satrajit Chakrabarty, Nicolò Gennaro, Wolfgang Holler, Manpreet Kaur, Pamela LaMontagne, MingDe Lin, Jan Lost, Daniel S. Marcus, R Maresca, Sarah Merkaj, Gabriel Cassinelli Pedersen, Marc von Reppert, Aristeidis Sotiras, Oleg M. Teytelboym, Niklas Tillmans, Malte Westerhoff, Ayda Youssef, Devon Godfrey, Scott Floyd, Andreas M. Rauschecker, Javier Villanueva-Meyer, Irada Pflüger, Jaeyoung Cho, Martin Bendszus, Gianluca Brugnara, Gloria J. Guzman Perez-Carillo, Derek R. Johnson, Anthony KamarXiv2025 · DOI · arXiv
Brain metastatic disease remains a significant complication of primary cancer, often associated with unfavorable prognoses. Improving diagnosis and treatment response assessment requires a shift toward automated AI-based volumetric segmentation of MRI images. However, the lack of high-quality, standardized, and multi-institutional annotated data has hindered clinical adoption. The BraTS-METS 2025 Lighthouse Challenge addresses this gap by providing a comprehensive dataset of 1,778 multi-parametric MRI cases, including longitudinal post-treatment scans. A central feature of this challenge is the "Lighthouse" benchmark, designed to quantify human inter-rater and intra-rater variability. By establishing these benchmarks, the challenge seeks to validate AI tools for rapid volumetric segmentation, ultimately improving personalized patient care in neuro-oncology.
Introduction
Brain metastases (BM) are the most frequent central nervous system tumors in adults. In current clinical practice, the gold standard for assessing treatment response—such as following stereotactic radiosurgery (SRS)—is the RANO-BM (Response Assessment in Neuro-Oncology Brain Metastases) criteria. This method relies on unidimensional (1D) measurements of the longest tumor diameter. However, 1D measurements often fail to capture the complex, asymmetrical morphological changes tumors undergo during treatment.
While 3D volumetric measurements provide a more accurate representation of tumor burden and response, manual slice-by-slice segmentation is too labor-intensive for routine clinical workflows. Artificial intelligence (AI) offers a solution, but robust models require diverse, high-quality datasets that reflect the complexities of both pre-treatment and post-treatment imaging.
The BraTS-METS 2025 Lighthouse Challenge
The 2025 edition of the BraTS-METS challenge builds upon previous iterations by significantly expanding the dataset and introducing a focus on longitudinal tracking. The challenge aims to:
Provide a massive, multi-institutional dataset for training and testing.
Incorporate post-treatment MRI scans to account for treatment-induced changes.
Quantify human expert variability to establish a realistic "performance ceiling" for AI models.
Methodology and Data Pipeline
The challenge utilizes a standardized multi-parametric MRI protocol, including pre-contrast T1, post-contrast T1 (T1c), and T2-FLAIR sequences. To ensure data consistency across the eight participating institutions (including UCSF, UCSD, Yale, and Duke), all scans undergo a rigorous preprocessing pipeline:
Standardization: Conversion to NIfTI format and resampling to 1mm³ isotropic resolution.
Registration: Co-registration to the SRI24 anatomical atlas.
Skull-Stripping: Automated removal of non-brain tissue using deep learning models.
Annotation Strategy
The ground truth labels are generated using a 4-label system: Enhancing Tumor (ET), Non-Enhancing Tumor Core (NETC), Surrounding Non-Enhancing FLAIR Hyperintensity (SNFH/edema), and Resection Cavity (RC). To ensure high-quality annotations, a 5-step pipeline is employed, involving AI pre-segmentation (nnU-Net), manual refinement by medical students and neuroradiologists, automated quality control, and final approval by senior neuroradiologists.
The Lighthouse Benchmark: Quantifying Human Variability
A unique contribution of the 2025 challenge is the "Lighthouse" dataset—a subset of 75 cases annotated four separate times. Two independent board-certified neuroradiologists performed segmentations both "from scratch" and via "AI-refinement" across two sessions separated by a 7-day washout period. This rigorous approach allows the challenge to measure:
Inter-rater variability: Disagreement between different experts.
Intra-rater variability: Disagreement by the same expert over time.
AI-Assistance Impact: How pre-segmentation influences human speed and accuracy.
Dataset Composition
The total dataset comprises 1,778 cases, making it one of the largest repositories of its kind for brain metastasis research.
Pre-treatment: 1,046 cases.
Post-treatment: 732 cases.
Data Splits: 1,296 cases for training, 179 for validation, and 303 for the hidden test set.
Evaluation and Clinical Significance
Participants submit their models via the MLCommons MedPerf framework using Docker containers. Models are evaluated using the Dice Similarity Coefficient (DSC) and Normalized Surface Distance (NSD), alongside sensitivity and precision metrics.
The ultimate goal is to transition neuro-oncology from 1D diameter checks to automated 3D volumetric tracking. This shift will enable clinicians to detect subtle changes in tumor volume earlier, allowing for more responsive and personalized treatment adjustments.
Limitations and Future Directions
A primary challenge in post-treatment imaging is differentiating between true tumor recurrence and radiation necrosis (pseudo-progression). Currently, standard MRI sequences struggle to distinguish these entities. Future iterations of the challenge plan to incorporate clinical metadata, such as radiation dosage and timing, to help models learn these critical distinctions. The BraTS-METS Consortium continues to work toward the direct translation of these open-source AI tools into clinical software.
Summary updated
Abstract
Despite continuous advancements in cancer treatment, brain metastatic disease remains a significant complication of primary cancer and is associated with an unfavorable prognosis. One approach for improving diagnosis, management, and outcomes is to implement algorithms based on artificial intelligence for the automated segmentation of both pre- and post-treatment MRI brain images. Such algorithms rely on volumetric criteria for lesion identification and treatment response assessment, which are still not available in clinical practice. Therefore, it is critical to establish tools for rapid volumetric segmentations methods that can be translated to clinical practice and that are trained on high quality annotated data. The BraTS-METS 2025 Lighthouse Challenge aims to address this critical need by establishing inter-rater and intra-rater variability in dataset annotation by generating high quality annotated datasets from four individual instances of segmentation by neuroradiologists while being recorded on video (two instances doing "from scratch" and two instances after AI pre-segmentation). This high-quality annotated dataset will be used for testing phase in 2025 Lighthouse challenge and will be publicly released at the completion of the challenge. The 2025 Lighthouse challenge will also release the 2023 and 2024 segmented datasets that were annotated using an established pipeline of pre-segmentation, student annotation, two neuroradiologists checking, and one neuroradiologist finalizing the process. It builds upon its previous edition by including post-treatment cases in the dataset. Using these high-quality annotated datasets, the 2025 Lighthouse challenge plans to test benchmark algorithms for automated segmentation of pre-and post-treatment brain metastases (BM), trained on diverse and multi-institutional datasets of MRI images obtained from patients with brain metastases.
BibTeX
@misc{maleki2025analysis,
author = {Nazanin Maleki and Raisa Amiruddin and Ahmed W. Moawad and Nikolay Yordanov and Athanasios Gkampenis and Pascal Fehringer and Fabian Umeh and Crystal Chukwurah and Fatima Memon and Bojan Petrović and Justin Cramer and Mark Krycia and Elizabeth B. Shrickel and Ichiro Ikuta and Gerard Thompson and Lorenna Vidal and Vilma Kosović and Adam Goldman-Yassen and Virginia Hill and Tiffany Y. So and Sedra Abou Ali Mhana and Abdullah Alotaibi and Nathan Page and Prateek Bhatia and Melisa S. Guelen and Yasaman Sharifi and Marko Jakovljevic and Salma AS Abosabie and Sara A. Abosabie and Mohanad Ghonim and Mohamed Ghonim and Amirreza Manteghinejad and Anastasia Janas and Kiril Krantchev and Maruf Adewole and Jake Albrecht and Udunna Anazodo and Sanjay Aneja and Anwar, Syed Muhammad and Timothy Bergquist and Veronica Chiang and Verena Chung and Gian Marco Conte and Farouk Dako and J. Mark Eddy and Ivan Ezhov and Khalili, Nastaran and Keyvan Farahani and Juan Eugenio Iglesias and Zhifan Jiang and Elaine Johanson and Anahita Fathi Kazerooni and Florian Kofler and Dominic LaBella and Koen Van Leemput and Hongwei Li and Marius George Linguraru and Xinyang Liu and Zeke Meier and Bjoern Menze and Harrison Moy and Klara Osenberg and Marie Piraud and Zachary Reitman and Russell Takeshi Shinohara and Chunhao Wang and Benedikt Wiestler and Walter F. Wiggins and Umber Shafique and Klara Willms and Arman Avesta and Khaled Bousabarah and Satrajit Chakrabarty and Nicolò Gennaro and Wolfgang Holler and Manpreet Kaur and Pamela LaMontagne and MingDe Lin and Jan Lost and Daniel S. Marcus and R Maresca and Sarah Merkaj and Gabriel Cassinelli Pedersen and Marc von Reppert and Aristeidis Sotiras and Oleg M. Teytelboym and Niklas Tillmans and Malte Westerhoff and Ayda Youssef and Devon Godfrey and Scott Floyd and Andreas M. Rauschecker and Javier Villanueva-Meyer and Irada Pflüger and Jaeyoung Cho and Martin Bendszus and Gianluca Brugnara and Gloria J. Guzman Perez-Carillo and Derek R. Johnson and Anthony Kam},
title = {{Analysis of the MICCAI Brain Tumor Segmentation -- Metastases (BraTS-METS) 2025 Lighthouse Challenge: Brain Metastasis Segmentation on Pre- and Post-treatment MRI}},
year = {2025},
eprint = {2504.12527},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2504.12527},
howpublished = {arXiv},
}