Directional Sinogram Inpainting for Limited Angle Tomography

Robert Tovey, Martin Benning, Christoph Brüne, Marinus J. Lagerwerf, Sean M. Collins, Rowan K. Leary, Paul A. Midgley, Carola‐Bibiane SchönliebApollo 2018 · DOI

TL;DR

This paper introduces a novel joint model for reconstructing 3D volumes from limited-angle X-ray tomography data. By simultaneously inpainting missing sinogram data and reconstructing the volume, their method significantly reduces characteristic artifacts and recovers more accurate structural information compared to existing state-of-the-art techniques.

What Problem Does This Paper Address?

Many applications of X-ray tomography, such as in electron microscopy and mammography, face a "limited angle problem." This arises when physical constraints prevent the X-ray source from rotating a full 180 degrees around the sample. This incomplete data leads to characteristic artifacts in the reconstructed 3D volume, often appearing as streak artifacts or blurred boundaries. These artifacts can obscure crucial details, hindering accurate analysis. Traditional methods struggle to effectively address both the missing data (inpainting) and the reconstruction simultaneously, often treating them as separate problems or relying on priors that are insufficient for severe data limitations.

Methods

The core of this work is a joint optimization framework that simultaneously reconstructs the 3D volume ($u$) and inpaints the missing sinogram data ($v$). This is formulated as minimizing a non-convex and non-smooth energy functional:

$E(u, v) = \alpha_1 ||Ru - v||^2 + \alpha_2 ||SRu - b||^2 + \alpha_3 ||Sv - b||^2 + \beta_1 TV(u) + \beta_2 DTV_{A_r}(v)$

Here:

  • $R$ is the X-ray transform, $S$ is a sampling mask, and $b$ is the noisy, limited-angle data.
  • $|| \cdot ||$ denotes appropriate norms (e.g., $L^2$ for data fidelity, $L^1$ for noise robustness).
  • $TV(u)$ is the standard Total Variation (TV) regularizer, promoting piecewise constant image intensities.
  • $DTV_{A_r}(v)$ is a novel directional Total Variation (DTV) regularizer applied to the sinogram. This is the key innovation, using a structure tensor ($A_r$) to adaptively penalize gradients. It allows for preserving sharp features in specific directions while smoothing in others, crucial for extrapolating missing sinogram data.

The minimization is achieved using an alternating (block) descent framework. This approach iteratively updates $u$ and $v$ while keeping the other fixed, leveraging linearized versions of the DTV term to maintain convexity in each sub-problem, making it computationally tractable.

Datasets

The authors evaluate their method on:

  • Two synthetic datasets:
    • A "Canonical Synthetic Dataset" of concentric rings, designed to be radially symmetric and thus trivialize directional inpainting.
  • A "Non-Trivial Synthetic Dataset" based on the modified Shepp-Logan phantom, offering a more complex geometry.
  • One experimental dataset: Acquired from a Transmission Electron Microscopy (TEM) experiment on a silver bipyramidal crystal. This dataset presents real-world challenges like sample holder artifacts and carbon coating.

Key Insights and Ideas

  • Joint Inpainting and Reconstruction is Crucial: Treating inpainting and reconstruction as separate steps is insufficient for severe limited-angle problems. A joint approach allows information from the reconstruction to guide the inpainting and vice-versa, leading to more coherent results.
  • Directional TV for Sinogram Regularization: The novel directional TV regularizer is key. By adapting to local image structure, it can intelligently extrapolate missing sinogram data, preserving important features and avoiding the isotropic blurring of standard TV.
  • Artifact Reduction Beyond Standard TV: While standard TV regularization improves reconstructions over simpler methods like FBP and SIRT, it still struggles with severe missing wedges. The proposed DTV approach demonstrably outperforms standard TV in these challenging scenarios.
  • Robustness to Model Violations: The method shows promising results even when the underlying X-ray transform model is violated, as seen in the experimental TEM data, suggesting practical applicability.

Key Results

  • Qualitative Improvement: The proposed method consistently produces reconstructions with cleaner structures and sharper boundaries compared to TV, FBP, and SIRT, especially in regions affected by the missing wedge.
  • Quantitative Improvements: While specific PSNR/SSIM values are noted as inconclusive for the experimental data, the authors report consistent improvements in recovering global geometry and reducing artifacts. For the canonical synthetic dataset, the method accurately extrapolates local structures to recover global geometry.
  • Effective Inpainting: The directional TV regularizer effectively inpaints missing sinogram data, allowing for more accurate reconstruction of the underlying object.

Limitations and Future Work

  • Hyperparameter Tuning: The method involves several hyperparameters that require careful tuning, although the authors note some insensitivity around optimal values. They suggest that a gridsearch might yield better results but would mask interpretability.
  • Computational Complexity: While the alternating descent framework makes the problem tractable, the complexity of the directional TV term can still be significant.
  • Future Directions:
    • Applying the framework to other imaging problems with missing data or occlusions (e.g., heavy metal artifacts).
  • Exploring alternative numerical algorithms for faster convergence or better avoidance of local minima in the non-convex landscape.
  • Developing more advanced sinogram regularizers that can better handle both visible and invisible singularities without compromising visible regions.

Summary updated

Abstract

In this paper we propose a new joint model for the reconstruction of tomography data under limited angle sampling regimes. In many applications of Tomography, e.g. Electron Microscopy and Mammography, physical limitations on acquisition lead to regions of data which cannot be sampled. Depending on the severity of the restriction, reconstructions can contain severe, characteristic, artefacts. Our model aims to address these artefacts by inpainting the missing data simultaneously with the reconstruction. Numerically, this problem naturally evolves to require the minimisation of a non-convex and non-smooth functional so we review recent work in this topic and extend results to fit an alternating (block) descent framework. \oldtext{We illustrate the effectiveness of this approach with numerical experiments on two synthetic datasets and one Electron Microscopy dataset.} \newtext{We perform numerical experiments on two synthetic datasets and one Electron Microscopy dataset. Our results show consistently that the joint inpainting and reconstruction framework can recover cleaner and more accurate structural information than the current state of the art methods.
BibTeX
@inproceedings{tovey2018directional,
  author = {Robert Tovey and Martin Benning and Christoph Brüne and Marinus J. Lagerwerf and Sean M. Collins and Rowan K. Leary and Paul A. Midgley and Carola‐Bibiane Schönlieb},
  title = {{Directional Sinogram Inpainting for Limited Angle Tomography}},
  booktitle = {Apollo},
  year = {2018},
  doi = {10.17863/cam.33373},
}
Figure