Implicit Neural Representations for Robust Joint Sparse-View CT Reconstruction

Jiayang Shi, Junyi Zhu, Daniel M. Pelt, K. Joost Batenburg, Matthew B. Blaschko

TL;DR

The authors introduce INR-Bayes, a novel Bayesian framework that improves sparse-view CT reconstruction by jointly reconstructing multiple similar objects using Implicit Neural Representations (INRs). By incorporating latent variables that capture shared statistical regularities across different scans, the method significantly enhances image quality, mitigates noise, and solves the notorious overfitting problem common to coordinate-based neural networks.

What Problem Does This Paper Address?

Computed Tomography (CT) is indispensable for medical diagnostics and industrial quality control, but standard scans expose subjects to ionizing radiation. While "sparse-view" CT reduces radiation by taking fewer X-ray projections, it introduces a highly under-sampled, ill-posed inverse problem.

Recently, Implicit Neural Representations (INRs)—which map continuous spatial coordinates to object densities using Multi-Layer Perceptrons (MLPs)—have emerged as a powerful alternative to traditional pixel-grid reconstructions. However, training an INR on a single sparse-view scan often leads to severe overfitting and artifact generation because the network lacks sufficient data constraints.

Existing methods attempt to solve this by sharing information across multiple scans (e.g., using meta-learning frameworks like MAML or FedAvg), but these approaches primarily focus on accelerating convergence speed rather than maximizing final image quality. During the individual adaptation phase, these methods tend to discard the learned prior information, leading back to the same overfitting issues.

Methods

INR-Bayes addresses this gap by treating joint reconstruction through a principled Bayesian lens. Instead of optimizing each object's INR independently, the framework links them via a shared, dynamic global prior distribution in the network parameter space.

Compared to individual reconstruction, joint reconstruction enables INRs to share statistical regularities among multiple objects.

The Variational Expectation-Maximization (VEM) Framework

The framework models the weights $w_j$ of the $j$-th object's INR as being sampled from a global, axis-aligned multivariate Gaussian prior $\mathcal{N}(\omega, \sigma)$, where $\omega$ and $\sigma$ are latent variables representing the collective mean and variance across all reconstruction nodes.

Optimization proceeds via a two-step iterative process:

  1. E-step (Individual Update): Each individual network optimizes its weight distribution parameters $(\mu_j, \rho_j)$ independently. The loss function balances data fidelity (matching the sparse X-ray measurements) with a Kullback-Leibler (KL) divergence penalty that prevents the individual weights from straying too far from the global prior.
  2. M-step (Global Update): The latent variables $\{\omega, \sigma\}$ are updated using a closed-form solution derived from the average and dispersion of all individual network weights.
The framework uses latent variables to capture the relation among all reconstruction nodes.

To make the expected likelihood tractable during gradient descent, the authors employ the reparameterization trick alongside Monte Carlo sampling.

Datasets

The authors evaluated their approach across a diverse set of datasets to prove its broad applicability:

  • WalnutCT: 42 high-resolution CT volumes of walnuts, representing industrial quality control scenarios.
  • AluminumCT: 25 scans of an aluminum alloy at various fatigue-corrosion phases.
  • LungCT: Medical scans from 96 patients via the Medical Segmentation Decathlon.
  • 4DCT: A dataset capturing 10 distinct temporal respiratory phases of a human lung.
  • CelebA: A natural image dataset of celebrity faces, included purely to demonstrate the framework's out-of-domain generalizability.

Key Insights and Ideas

  • Regularization in Weight Space, Not Image Space: Traditional joint reconstruction methods often enforce similarity in the image domain (e.g., total variation or shared features). INR-Bayes shifts this paradigm by enforcing statistical regularities directly within the neural network's parameter space.
  • Self-Adjusting Regularization Strength: Because the global prior estimates variance ($\sigma$) element-wise, the framework automatically adjusts its regularization strength. Weight components with high variance across the population are allowed more freedom to diverge, while low-variance components are strictly constrained to match the group consensus.
  • An Antidote to INR Overfitting: Standard INRs suffer from performance degradation if trained for too many iterations on sparse data. By anchoring the individual networks to a dynamically updated global prior, INR-Bayes maintains stable, high-quality reconstructions even during extended training runs.

Key Results

INR-Bayes consistently outperformed classical baselines (FBP, SIRT), regularization techniques (RegTGV, RegLLT-TV), and competing joint INR methods (MAML, FedAvg, INRWild) across both noiseless and noisy measurement setups.

  • Reconstruction Quality: On the Inter-walnut dataset, INR-Bayes achieved a Peak Signal-to-Noise Ratio (PSNR) of 36.13 dB, outperforming SingleINR (35.18 dB) and MAML (35.66 dB).
  • Robustness to Noise: Under simulated Poisson noise, the baseline methods rapidly deteriorated due to overfitting as training progressed. INR-Bayes maintained stable, high-quality performance, achieving a noisy lung reconstruction PSNR of 31.20 dB compared to SingleINR's 27.16 dB.
  • Generalization to Unseen Data: The learned global prior mean ($\omega$) can be extracted and used as a highly effective network initialization for entirely new, unscanned subjects, accelerating convergence and improving final accuracy.

Limitations and Future Work

Despite its strong performance, the authors note a few limitations:

  • Computational Overhead: INR-Bayes introduces approximately a 15% increase in training time compared to standard INR methods (taking ~22.7 minutes per node on an NVIDIA A100 GPU), primarily due to the Monte Carlo sampling and KL divergence calculations.
  • Clinical Validation: While numerical metrics like PSNR and SSIM showed significant improvements, these metrics do not always perfectly correlate with clinical diagnostic utility. Future work will require rigorous clinical verification by radiologists before deployment in medical environments.

Summary updated

Abstract

Computed Tomography (CT) is pivotal in industrial quality control and medical diagnostics. Sparse-view CT, offering reduced ionizing radiation, faces challenges due to its under-sampled nature, leading to ill-posed reconstruction problems. Recent advancements in Implicit Neural Representations (INRs) have shown promise in addressing sparse-view CT reconstruction. Recognizing that CT often involves scanning similar subjects, we propose a novel approach to improve reconstruction quality through joint reconstruction of multiple objects using INRs. This approach can potentially utilize the advantages of INRs and the common patterns observed across different objects. While current INR joint reconstruction techniques primarily focus on speeding up the learning process, they are not specifically tailored to enhance the final reconstruction quality. To address this gap, we introduce a novel INR-based Bayesian framework integrating latent variables to capture the common patterns across multiple objects under joint reconstruction. The common patterns then assist in the reconstruction of each object via latent variables, thereby improving the individual reconstruction. Extensive experiments demonstrate that our method achieves higher reconstruction quality with sparse views and remains robust to noise in the measurements as indicated by common numerical metrics. The obtained latent variables can also serve as network initialization for the new object and speed up the learning process.
BibTeX
@article{shi2024implicit,
  author = {Jiayang Shi and Junyi Zhu and Daniel M. Pelt and K. Joost Batenburg and Matthew B. Blaschko},
  title = {{Implicit Neural Representations for Robust Joint Sparse-View CT Reconstruction}},
  journal = {Accepted by TMLR},
  year = {2024},
  url = {https://openreview.net/forum?id=XCzuQI0oXR},
}
Figure