Learning from a Handful Volumes: MRI Resolution Enhancement with Volumetric Super-Resolution Forests

Aline Sindel, Katharina Breininger, Johannes Käßer, Andreas Hess, Andreas Maier, Thomas KöhlerarXiv 2018 · arXiv

TL;DR

Volumetric Super-Resolution Forests (VSRF) provide a fast, data-efficient way to enhance MRI resolution retrospectively. By using random forest regression to learn local mappings between 3D image patches, this method outperforms deep learning and dictionary-based approaches while requiring only a handful of training volumes and running efficiently on standard CPUs.

What Problem Does This Paper Address?

Magnetic Resonance Imaging (MRI) faces a fundamental trade-off between spatial resolution and scan time. High-resolution scans provide the fine anatomical detail necessary for early pathology detection, but they require long acquisition times, which increases patient discomfort and the likelihood of motion artifacts.

While Super-Resolution (SR) techniques can "up-sample" low-resolution (LR) scans into high-resolution (HR) versions, existing methods have drawbacks. Traditional interpolation is too blurry; dictionary-based methods are computationally expensive; and modern Deep Learning (CNN) approaches, while powerful, typically require massive amounts of training data that are often unavailable in specialized medical contexts. The authors seek a middle ground: a method that is accurate, fast, and works with very limited training data.

Methods

The authors propose Volumetric Super-Resolution Forests (VSRF), which generalizes 2D SR forest techniques to 3D volumetric data.

  1. Random Forest Regression: Instead of a black-box neural network, the system uses a forest of decision trees. During training, the trees recursively partition the data space based on 3D patch features.
  2. Locally Linear Mapping: In each leaf node, the model learns a specific linear mapping (via ridge regression) that transforms an LR patch into its HR counterpart.
  3. Custom 3D Features (DevEdge): To capture 3D structure, the authors use a set of 10 features: first- and second-order derivatives in all three directions ($x, y, z$), plus 3D edge magnitude and orientation.
  4. Median Ensemble: Standard random forests average the predictions of all trees. The authors found that using the median of the tree predictions is more robust, significantly reducing patch artifacts and outliers.
  5. Residual Learning: The forest doesn't predict the whole HR image from scratch. Instead, the LR volume is first upscaled via tricubic interpolation, and the forest predicts only the "missing" high-frequency details (the residual), which are then added back.

Datasets

The method was evaluated on two distinct datasets:

  • Mouse Brain Dataset: 21 isotropic high-resolution volumes acquired with a 4.7T scanner.
  • Kirby 21 Human Brain Dataset: 42 T1-weighted volumes acquired with a 3T scanner. This is a publicly available dataset used to benchmark reproducibility.

Key Insights and Ideas

  • Extreme Data Efficiency: One of the most striking findings is that VSRF performs remarkably well with very little data. As shown in Figure 4, performance jumps sharply with just 1 to 5 training volumes and plateaus around 13. This makes it highly practical for clinical settings where HR data is scarce.
Performance vs. Training Volumes
  • The Power of the Median: Switching from a mean to a median ensemble was a key "aha" moment. It provided a clear boost in both PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index), as it prevents a single "bad" tree prediction from blurring the final result.
Median vs. Average Ensemble
  • True 3D vs. Pseudo-3D: Many "3D" methods actually process 2D slices and average them (pseudo-3D). VSRF uses true 3D cubic patches, which better exploits the spatial correlations between adjacent slices, leading to sharper results in all anatomical planes.

Key Results

  • Quantitative Superiority: VSRF achieved the highest PSNR and SSIM scores across both datasets. For the mouse brain, it reached a PSNR of 39.46 dB, outperforming the next best method (VA+) by 0.71 dB.
  • Qualitative Sharpness: Visually, VSRF recovers fine structures like the arbor vitae (the branching white matter of the cerebellum) much more clearly than other methods, which often leave these areas blurry or introduce artifacts.
Visual Comparison of Results
  • Computational Efficiency: VSRF is highly efficient. Inference takes less than 1 minute per volume on a CPU. In contrast, a 3D CNN (SRCNN) required 3 days of training on a high-end GPU to achieve comparable results.

Limitations and Future Work

While VSRF is robust, it still relies on having at least a few high-resolution example volumes for the specific anatomy being scanned. The authors also note that while the method is fast, there is always room to optimize the feature extraction and PCA dimensionality reduction steps.

Future work could involve integrating this method directly into clinical workflows to reduce scan times in real-time or applying the volumetric forest approach to other modalities like CT or PET scans.

Summary updated

Abstract

Magnetic resonance imaging (MRI) enables 3-D imaging of anatomical structures. However, the acquisition of MR volumes with high spatial resolution leads to long scan times. To this end, we propose volumetric super-resolution forests (VSRF) to enhance MRI resolution retrospectively. Our method learns a locally linear mapping between low-resolution and high-resolution volumetric image patches by employing random forest regression. We customize features suitable for volumetric MRI to train the random forest and propose a median tree ensemble for robust regression. VSRF outperforms state-of-the-art example-based super-resolution in term of image quality and efficiency for model training and inference in different MRI datasets. It is also superior to unsupervised methods with just a handful or even a single volume to assemble training data.
BibTeX
@misc{sindel2018learning,
  author = {Aline Sindel and Katharina Breininger and Johannes Käßer and Andreas Hess and Andreas Maier and Thomas Köhler},
  title = {{Learning from a Handful Volumes: MRI Resolution Enhancement with Volumetric Super-Resolution Forests}},
  year = {2018},
  eprint = {1802.05518},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1802.05518},
}
Figure