Manifold reconstruction of differences: a model-based iterative statistical estimation algorithm with a data-driven prior

Matthew Tivnan, J. Webster StaymanarXiv 2021 · DOI · arXiv

Abstract

Manifold learning using deep neural networks has been shown to be an effective tool for building sophisticated prior image models that can be applied to noise reduction in low-dose CT. A manifold is a low-dimensional space that captures most of the variation in a class of data (e.g., chest CT images). This nonlinear reduction in dimensionality is one way to build up a sophisticated model of the features and structures present in image data. This nonlinear dimensionality reduction tends to learn features associated with the signal rather than the noise when provided with a large training dataset. However, the internal workings are typically not easy to understand. The results can be highly nonlinear and object-dependent. After training, the model is fixed, and during reconstruction, there is typically no check for fitness between the image estimates and the measured data. We propose a new iterative CT reconstruction algorithm, called Manifold Reconstruction of Differences (MRoD), which combines physical and statistical models with a data-driven prior based on manifold learning. The MRoD algorithm involves estimating a manifold component, approximating common features among all patients, and the difference component, which has the freedom to fit the measured data. By applying a sparsity-promoting penalty to the difference image rather than a hard constraint to the manifold, the MRoD algorithm is able to reconstruct features which are not present in the training data. The difference component itself may be independently useful. While the manifold captures typical patient features (e.g., healthy anatomy), the difference image highlights patient-specific elements (e.g., pathology). In this work, we present a description of an optimization framework which combines trained manifold-based modules with physical modules. We present a simulation study using anthropomorphic lung data showing that the MRoD algorithm can both isolate differences between a particular patient and the typical distribution, but also provide significant noise reduction with less bias than a typical penalized likelihood estimator in composite manifold plus difference reconstructions.

Introduction

Deep learning has revolutionized image denoising, particularly in low-dose CT where noise is a major hurdle. Most current methods use manifold learning, where a neural network learns a low-dimensional "map" of what a typical human scan looks like. However, there is a dangerous catch: if a patient has a rare condition—like a specific type of tumor or a tiny bone fracture—that wasn't in the network's training data, the model might treat that pathology as "noise" and smooth it away to make the image fit the learned manifold. Conversely, it might "hallucinate" healthy structures where they don't exist. Furthermore, many deep learning models lack an explicit "sanity check" to ensure the final image actually matches the raw X-ray data measured by the scanner.

Methodology

The authors introduce MRoD, which treats reconstruction as a joint optimization problem. Instead of forcing the final image to lie strictly on a learned manifold, they model the image ($x$) as the sum of two parts:

  1. The Manifold Component ($D(m)$): This is generated by a pre-trained deep neural network (a convolutional autoencoder). It represents the "typical" anatomy common across patients.
  2. The Difference Component ($x_D$): This is a sparse image that captures anything the manifold missed.
The convolutional autoencoder architecture used to generate the manifold component.

The algorithm minimizes a cost function that balances two things:

  • Data Fidelity: A physics-based term ensuring the sum of the two components matches the actual CT measurements.
  • Sparsity Penalty: An L1-norm penalty on the difference component. This encourages the algorithm to use the "typical" manifold as much as possible, only adding to the "difference" image when the raw data demands it.

By using an iterative approach (the Adam optimizer), the system "negotiates" between what the deep learning model expects to see and what the X-ray physics says is actually there.

Experimental Setup

The study used the XCAT 2.0 anthropomorphic digital phantom, a sophisticated tool for generating realistic simulated human anatomy.

  • Training: 100 simulated patients (2D lung slices) were used to train the DNN encoder-decoder.
  • Testing: Three additional simulated patients were used for evaluation. To test the "out-of-distribution" performance, the researchers manually added pathological features—a spiculated lung nodule and a 4mm rib crack—that were never seen by the network during training.

Results and Discussion

MRoD demonstrates a significant improvement in the bias-noise tradeoff. When compared to standard Quadratic Penalized Likelihood (QPL) at a matched noise level of 30 HU, MRoD showed significantly less bias. While QPL resulted in spatial blurring, MRoD maintained sharp edges.

One of the most compelling ideas is that the "difference component" isn't just a mathematical leftover; it is clinically meaningful. Because the manifold component captures "normal" anatomy, the difference image naturally highlights anomalies like tumors or fractures, potentially acting as a built-in "heat map" for radiologists.

Comparison between the full MRoD reconstruction (left) and the isolated difference component (right), highlighting patient-specific features.

In the "cracked rib" and "lung nodule" tests, the manifold component (the DL part) failed to show the injuries because it hadn't learned them. However, the difference component successfully captured these features, ensuring they appeared clearly in the final composite image. This "soft constraint" approach allows the image to look like the training data unless the physics proves otherwise.

Conclusion

The MRoD algorithm successfully combines the pattern recognition power of deep learning with the physical truth of statistical models. By splitting the image into manifold and difference components, it achieves high-quality noise reduction without sacrificing rare or out-of-distribution pathological features. Future work will focus on extending this framework to 3D clinical data and optimizing computational costs for real-time application.

Summary updated

Abstract

Manifold learning using deep neural networks been shown to be an effective tool for building sophisticated prior image models that can be applied to noise reduction in low-close CT. A manifold is a low-dimensional space that captures most of the variation in a class of data (e.g. chest CT images). This nonlinear reduction in dimensionality is one way to build up a sophisticated model of the features and structures present in image data. This nonlinear dimensionality reduction tends to learn features associated with the signal rather than the noise when provided with a large training dataset. However, the internal workings operation are typically not easy to understand. The results can be highly nonlinear and object-dependent. After training the model is fixed and during reconstruction is typically no check for fitness between the image estimates and the measured data. We propose a new iterative CT reconstruction algorithm, called Manifold Reconstruction of Differences (MRoD), which combines physical and statistical models with a data-driven prior based on manifold learning. The MRoD algorithm involves estimating a <i>manifold component</i>, approximating common features among all patients, and the <i>difference component</i> which has the freedom to fit the measured data. By applying a sparsity-promoting penalty to the difference image rather than a hard constraint to the manifold, the MRoD algorithm is able to reconstruct features which are not present in the training data. The difference component itself may be independently useful. While the manifold captures typical patient features (e.g. healthy anatomy), the difference image highlights patient-specific elements (e.g. pathology). In this work, we present a description of an optimization framework which combines trained manifold-based modules with physical modules. We present a simulation study using anthropomorphic lung data showing that the MRoD algorithm can both isolate differences between a particular patient and the typical distribution, but also provide significant noise reduction with less bias than a typical penalized likelihood estimator in composite manifold plus difference reconstructions.
BibTeX
@misc{tivnan2021manifold,
  author = {Matthew Tivnan and J. Webster Stayman},
  title = {{Manifold reconstruction of differences: a model-based iterative statistical estimation algorithm with a data-driven prior}},
  year = {2021},
  eprint = {2010.09685},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2010.09685},
}
Figure