TL;DR
This paper introduces a novel benchmark database of diffusion MRI data from the same subjects acquired across three different scanners and multiple protocols. They then evaluate five advanced harmonisation algorithms, demonstrating that while significant progress has been made in reducing inter-scanner variability, challenges remain in achieving high-fidelity spatial and angular resolution enhancement.
What Problem Does This Paper Address?
Diffusion MRI (dMRI) is a powerful tool for probing brain microstructure, but its quantitative measures are highly sensitive to variations in scanner hardware, acquisition protocols, and even the time of day. This variability makes it difficult to combine data from different research sites or studies, limiting the statistical power and generalizability of findings. While harmonisation techniques aim to mitigate these differences, a standardized benchmark for evaluating their performance across diverse scanner and protocol conditions has been lacking. This paper addresses this gap by creating a comprehensive dataset and rigorously comparing existing harmonisation algorithms.
Methods
The core of this work lies in two main areas:
- Database Creation: The researchers collected dMRI data from 14 healthy volunteers on three different 3T MRI scanners with varying maximum gradient strengths (40, 80, and 300 mT/m). For the higher-gradient scanners, they acquired both a "standard" protocol (matched to the lower-gradient scanner) and a "state-of-the-art" protocol with improved spatial and angular resolution. This multi-scanner, multi-protocol approach provides a rich testbed for harmonisation.
- Algorithm Evaluation: Five advanced harmonisation algorithms (SHNet, SHResNet, SphericalNet, FCSNet, and SDL) were evaluated on two tasks:
- Scanner-to-scanner mapping at matched resolution: Harmonizing data from different scanners acquired with similar protocols.
- Spatial and angular resolution enhancement: Improving the resolution of lower-quality data to match that of state-of-the-art acquisitions. The performance of these algorithms was assessed using various metrics (FA, MD, RO, R2) at global, regional, and local levels, comparing them against a simple trilinear interpolation "reference."
Datasets
The primary dataset introduced in this paper is a benchmark diffusion MRI database comprising data from 14 healthy volunteers. This database features:
- Acquisitions on three distinct 3T scanners (40, 80, and 300 mT/m gradient strength).
- Two acquisition protocols per scanner (where applicable): a "standard" (ST) protocol and a "state-of-the-art" (SA) protocol with higher spatial and angular resolution.
- Both training and testing subsets were defined for algorithm evaluation.
Key Insights and Ideas
- The "Travelling Head" Approach is Crucial: Using the same subjects scanned sequentially across different scanners is a powerful way to isolate scanner- and protocol-specific variability from inter-subject differences.
- Deep Learning Dominates, But Not Universally: While deep learning-based methods (SHNet, SHResNet, SphericalNet, FCSNet) generally outperformed traditional interpolation methods, their performance varied significantly across different metrics and tasks.
- Anisotropy is Harder to Harmonize: Metrics related to anisotropy (like FA and R2) proved more challenging to harmonize accurately compared to more isotropic measures (like MD and RO), likely due to their sensitivity to subtle structural differences and alignment issues.
- Resolution Enhancement is a Bigger Hurdle: Harmonizing data to improve spatial and angular resolution was significantly more difficult than simply matching resolution across scanners, highlighting the limitations of current techniques in upscaling data fidelity.
- Preprocessing Matters: The paper emphasizes that harmonisation is sensitive to preprocessing steps, and inconsistencies can introduce additional variability that is hard to correct post-hoc.
Key Results
- Scanner-to-Scanner Harmonization: The best performing algorithms significantly reduced variability between scanners, with some approaching the reproducibility levels of scan-rescan experiments for metrics like MD and R2. For example, median localized MNE (mean normalized error) was often below 5% for harmonized data, compared to higher values for the reference interpolation.
- Resolution Enhancement: This task showed poorer performance. While some algorithms improved upon the reference, errors remained substantial, with localized MNE often exceeding 15% for anisotropy metrics.
- Algorithm Performance: FCSNet emerged as a consistently strong performer across various metrics and tasks, particularly in scanner-to-scanner mapping. SHResNet and SphericalNet also showed competitive results.
- Scan-Rescan Comparison: The best harmonisation methods achieved MSE values comparable to scan-rescan differences, suggesting they can effectively reduce systematic scanner biases.
Limitations and Future Work
- Limited Sample Size: While 14 subjects are valuable for a benchmark, larger cohorts would be needed for more robust statistical validation and to assess the impact on real-world group studies.
- Focus on Single-Shell: The current evaluation primarily focused on single-shell dMRI data. Future work should extend to multi-shell data, which is becoming increasingly common.
- Preprocessing Impact: The paper acknowledges that variations in preprocessing pipelines can influence harmonisation results, and further investigation into optimal preprocessing strategies is needed.
- Gradient Nonlinearity: The impact of gradient nonlinearity on the 300 mT/m system was noted as a potential source of increased variance, and this information will be made available for future research.
- Clinical Relevance: While the study provides a strong foundation, further work is needed to assess the impact of harmonisation on actual clinical analyses, such as differentiating patient groups from controls. The authors suggest exploring the impact on tractography and group-level analyses.
Summary updated