Diffusion MRI microstructure models with in vivo human brain Connectom data: results from a multi-group comparison

Uran Ferizi, Benoît Scherrer, Torben Schneider, Mohammad Alipoor, Odin Eufracio, Rutger Fick, Rachid Deriche, Markus Nilsson, Ana K. Loya-Olivas, Mariano Rivera, Dirk H. J. Poot, Alonso Ramírez-Manzanares, José L. Marroquín, Ariel Rokem, Christian De Potter, Robert F. Dougherty, Ken Sakaie, Claudia A. M. Gandini Wheeler‐Kingshott, Simon K. Warfield, Thomas Witzel, Lawrence L. Wald, José G. Raya, Daniel C. AlexanderarXiv 2016 · arXiv

TL;DR

This paper presents the results of the "White Matter Modeling Challenge," a competition designed to compare various mathematical models used to interpret diffusion MRI data. The challenge revealed that simpler, biophysical "tissue models" often outperform more complex "signal models" in predicting unseen data, and that removing signal outliers can improve performance.

What Problem Does This Paper Address?

Diffusion-weighted Magnetic Resonance Imaging (DW-MRI) is a powerful tool for non-invasively studying the brain's white matter microstructure. However, a vast array of mathematical models exist to interpret the complex signals generated by DW-MRI. These models range from those that describe the physical properties of tissue (tissue models) to those that focus solely on fitting the observed signal (signal models). Despite the proliferation of these models, there's a significant lack of standardized, head-to-head comparisons on real-world human brain data. This makes it difficult to determine which models are most accurate, robust, and ultimately useful for clinical applications. This paper addresses this gap by organizing a challenge to rigorously compare a diverse set of DW-MRI models.

Methods

The core of this paper is the "White Matter Modeling Challenge." Participants were given three-quarters of a rich, in-vivo DW-MRI dataset and tasked with building models to predict the remaining quarter. The models were evaluated based on their ability to accurately predict the unseen signal, using the sum of squared differences (SSE) as the primary metric, corrected for Rician noise. The challenge encouraged a wide variety of modeling approaches, including different noise assumptions, fitting algorithms, and parameter estimation strategies. The paper also provides detailed descriptions of each competing model in an appendix.

Datasets

The study utilized a high-quality, in-vivo DW-MRI dataset acquired from a single healthy volunteer using the Human Connectome scanner. This scanner is notable for its high gradient strength (up to 300 mT/m), allowing for a very rich sampling of the diffusion measurement space. The dataset was split into a training set (three-quarters) and a testing set (one-quarter). The data was acquired across two regions of interest: the genu of the corpus callosum (relatively straight fibers) and the fornix (more complex fiber architecture).

Key Insights and Ideas

  • Tissue Models Shine: Surprisingly, biophysical "tissue models" (which explicitly describe tissue microstructure) generally outperformed more flexible "signal models" (which focus on fitting the signal without explicit biophysical assumptions). This challenges the common perception that signal models offer superior flexibility.
  • Overfitting is a Risk: The superior performance of tissue models suggests that the flexibility of signal models can lead to overfitting the training data, hindering their ability to generalize to unseen data.
  • Noise Modeling Nuances: Assuming a non-Gaussian noise model provided little benefit over a Gaussian assumption, likely due to the high signal-to-noise ratio (SNR) of the acquired data.
  • Outlier Removal Matters: Preprocessing the training data by removing signal outliers significantly improved model performance for some models and was a common strategy among the top performers.
  • Prediction vs. Explanation: While models that predict unseen data well are valuable for rejecting poor models, this doesn't automatically mean they are mechanistically closer to the true tissue structure. Further validation is needed.

Key Results

The challenge involved 14 distinct models from various research groups. The top-ranking models, which demonstrated the best prediction accuracy, were primarily tissue models. The paper highlights that the ranking was largely dominated by performance on shells with b-values between 750 and 1400 s/mm². While specific quantitative performance metrics for each model are presented in figures and tables, the overarching finding is the consistent performance of certain tissue models across different regions and b-value ranges.

Limitations and Future Work

The authors acknowledge several limitations. The acquisition protocol was exceptionally rich and not representative of typical clinical scans, which might influence model performance. The study focused on isolated voxels, and future work could extend this to whole-brain analysis. The computational demand of models was not considered, which could impact their practical applicability. Future challenges could explore more typical clinical acquisition protocols and investigate the generalizability of findings across subjects. The authors also suggest that future work should focus on validating the parameters estimated by these models against independent measurements, such as histology, though they note the inherent difficulties in obtaining ground truth. The challenge data and model details are made publicly available to encourage further research and model development.

Summary updated

Abstract

A large number of mathematical models have been proposed to describe the measured signal in diffusion-weighted (DW) magnetic resonance imaging (MRI) and infer properties about the white matter microstructure. However, a head-to-head comparison of DW-MRI models is critically missing in the field. To address this deficiency, we organized the White Matter Modeling Challenge during the International Symposium on Biomedical Imaging (ISBI) 2015 conference. This competition aimed at identifying the DW-MRI models that best predict unseen DW data. in vivo DW-MRI data was acquired on the Connectom scanner at the A.A.Martinos Center (Massachusetts General Hospital) using gradients strength of up to 300 mT/m and a broad set of diffusion times. We focused on assessing the DW signal prediction in two regions: the genu in the corpus callosum, where the fibres are relatively straight and parallel, and the fornix, where the configuration of fibres is more complex. The challenge participants had access to three-quarters of the whole dataset, and their models were ranked on their ability to predict the remaining unseen quarter of data. In this paper we provide both an overview and a more in-depth description of each evaluated model, report the challenge results, and infer trends about the model characteristics that were associated with high model ranking. This work provides a much needed benchmark for DW-MRI models. The acquired data and model details for signal prediction evaluation are provided online to encourage a larger scale assessment of diffusion models in the future.
BibTeX
@misc{ferizi2016diffusion,
  author = {Uran Ferizi and Benoît Scherrer and Torben Schneider and Mohammad Alipoor and Odin Eufracio and Rutger Fick and Rachid Deriche and Markus Nilsson and Ana K. Loya-Olivas and Mariano Rivera and Dirk H. J. Poot and Alonso Ramírez-Manzanares and José L. Marroquín and Ariel Rokem and Christian De Potter and Robert F. Dougherty and Ken Sakaie and Claudia A. M. Gandini Wheeler‐Kingshott and Simon K. Warfield and Thomas Witzel and Lawrence L. Wald and José G. Raya and Daniel C. Alexander},
  title = {{Diffusion MRI microstructure models with in vivo human brain Connectom data: results from a multi-group comparison}},
  year = {2016},
  eprint = {1604.07287},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1604.07287},
}

Loading related content...

Figure