Reproducibility and quality assurance in MRI

Tony Stoecker, Kathryn E. Keenan, Florian Knöll, Nikos Priovoulos, Martin Uecker, Maxim ZaitsevMagnetic 2025 · DOI · OpenAlex

Abstract

Open Access funding enabled and organized by Projekt DEAL. This editorial introduces a special issue focused on bridging the reproducibility gap in Magnetic Resonance Imaging (MRI). By integrating metrology, open-source pipelines, and automated quality control, the field aims to transition MRI from a qualitative imaging tool into a precise, quantitative measurement device.

Introduction

Magnetic Resonance Imaging (MRI) is a cornerstone of modern medical diagnostics and neuroscience. However, the field currently faces a "reproducibility crisis." Scientific results often vary significantly across different scanners, software versions, and research teams. This editorial synthesizes the contributions of a special issue dedicated to overcoming these challenges.

The authors identify three primary culprits for the lack of reproducibility:

  1. The "Black Box" Problem: Reliance on proprietary, closed-source vendor software prevents researchers from verifying and sharing the exact parameters of their experimental setups.
  2. Lack of Standardization: Unlike fields such as radiotherapy, MRI often lacks a universal metrological framework. In many clinical contexts, scanners are treated as tools for generating images for subjective interpretation rather than as calibrated measurement devices.
  3. Technical Complexity: The vast array of options for data acquisition, reconstruction, and post-processing introduces numerous opportunities for bias and error.

Methodological Strategies for Standardization

To address these issues, the special issue proposes a multi-faceted approach involving metrology, open-source tools, and automated protocols.

Metrology and Phantoms

Integrating the "science of measurement" (metrology) into MRI ensures consistency across different sites and time points. This is facilitated by the use of standardized physical phantoms, such as those developed by ISMRM/NIST. These objects have known physical properties, allowing researchers to validate quantitative measurements like T1, T2, and Apparent Diffusion Coefficient (ADC) across hardware from multiple vendors. Furthermore, the democratization of phantoms through 3D printing and shared CAD files allows labs to create standardized testing tools cost-effectively.

Open-Source Pipelines

Transparency is fundamental to the "Three Rs" of scientific rigor: Repeatability (same team, same setup), Reproducibility (different team, same setup), and Replicability (different team, different setup). The editorial highlights the importance of open-source tools like Pulseq, which enables vendor-neutral pulse sequence design. By utilizing modular, open-source processing pipelines, researchers can ensure that every step of an experiment is transparent and reproducible by others.

Automated Quality Assurance

Moving away from manual, subjective image review is critical. The field is shifting toward objective, AI-driven, or phantom-based quality assurance (QA) and quality control (QC) protocols. These automated systems can detect "image quality drift" over time, providing a more reliable foundation for longitudinal and multi-center studies.

Key Insights and Quantitative Findings

The editorial summarizes several studies that demonstrate the practical impact of these standardized approaches:

  • Brain R1 Dispersion: In a study of 20 patients with small vessel disease, R1 dispersion measurements were found to be highly repeatable and capable of distinguishing healthy tissue from affected regions.
  • QSM Repeatability: Quantitative Susceptibility Mapping (QSM) remains a challenge. Test-retest data from healthy volunteers showed only "moderate" repeatability, which was highly sensitive to the specific reconstruction algorithm and region of interest (ROI) used.
  • Reference Standardization: Research indicated that the choice of anatomical reference regions in susceptibility mapping significantly impacts the statistical significance of clinical findings, highlighting the urgent need for standardized ROI referencing.

Discussion and Future Directions

The transition to a more reproducible MRI framework requires more than just technical solutions; it necessitates a cultural shift. Principal investigators, journals, and funding bodies must begin to reward rigorous reproducibility as much as the discovery of "novel" findings.

The primary remaining bottleneck is vendor lock-in. The reliance on closed-source software continues to hinder the "second and third Rs" (reproducibility and replicability). Future work is directed toward "vendor-neutral" acquisition and the implementation of automated, real-time quality control that can flag errors while the patient is still being scanned.

By treating the MRI scanner as a calibrated measurement device, the community can ensure that its findings are robust, reliable, and ultimately more beneficial for patient care.

Summary updated

Abstract

Open Access funding enabled and organized by Projekt DEAL.
BibTeX
@inproceedings{stoecker2025reproducibility,
  author = {Tony Stoecker and Kathryn E. Keenan and Florian Knöll and Nikos Priovoulos and Martin Uecker and Maxim Zaitsev},
  title = {{Reproducibility and quality assurance in MRI}},
  booktitle = {Magnetic},
  year = {2025},
  doi = {10.1007/s10334-025-01271-1},
}
Figure