Fourier Feature Network for 3D Vessel Reconstruction from Biplane Angiograms

Sean M. Wu, Naoki Kaneko, David S. Liebeskind, Fabien ScalzoResearch Square 2024 · DOI

Abstract

3D reconstruction of biplane cerebral angiograms remains a challenging, unsolved research problem due to the loss of depth information and the unknown pixelwise correlation between input images. The occlusions arising from only two views complicate the reconstruction of fine vessel details and the simultaneous addressing of inherent missing information. In this paper, we take an incremental step toward solving this problem by reconstructing the corresponding 2D slice of the cerebral angiogram using biplane 1D image data. We developed a coordinate-based neural network that encodes the 1D image data along with a deterministic Fourier feature mapping from a given input point, resulting in a slice reconstruction that is more spatially accurate. Using only one 1D row of biplane image data, our Fourier feature network reconstructed the corresponding volume slices with a peak signal-to-noise ratio (PSNR) of 26.32±0.36, a structural similarity index measure (SSIM) of 61.38±1.79, a mean squared error (MSE) of 0.0023±0.0002, and a mean absolute error (MAE) of 0.0364±0.0029. Our research has implications for future work aimed at improving backprojection-based reconstruction by first examining individual slices from 1D information as a prerequisite.

Introduction

Visualizing the brain's vascular system in 3D is a critical requirement for neurosurgical interventions, yet current imaging modalities present a significant trade-off between detail and safety. While 3D volumes from CT scans provide high resolution, they require hundreds of X-ray acquisitions, exposing patients to substantial ionizing radiation. Biplane angiography, which utilizes only two views (typically 0° and 90°), offers a safer alternative but results in a mathematically "underdetermined" reconstruction problem. The loss of depth and frequent vessel occlusions make it difficult for standard algorithms to resolve fine anatomical details. This research investigates a fundamental building block of this challenge: whether a single 2D cross-sectional slice can be accurately reconstructed using only one row of pixels from each of the two biplane views.

Methods

The proposed approach utilizes a Fourier Feature Network designed for 1D-to-2D reconstruction. Unlike traditional Convolutional Neural Networks (CNNs) that process entire images, this model employs a coordinate-based architecture. It takes a specific $(x, y)$ coordinate as input and predicts the corresponding pixel value at that location.

To overcome the "spectral bias" inherent in standard Multi-Layer Perceptrons (MLPs)—where networks favor smooth, low-frequency functions—the authors implemented Fourier Feature Mapping. Raw coordinates are mapped into a higher-dimensional space using a set of sine and cosine functions. This allows the network to capture the sharp, high-frequency transitions characteristic of vessel boundaries that would otherwise be blurred.

Overview of the Fourier Feature Network pipeline for 2D slice reconstruction from 1D biplane data.

The network is conditioned on 1D image data extracted from simulated biplane views. These rows are concatenated and passed through an encoder to provide global context for the slice reconstruction. Training was conducted using 45 high-quality 3D CT volumes, with biplane views simulated via Maximum Intensity Projection (MIP) to approximate clinical X-ray data.

Results and Discussion

The Fourier Feature Network demonstrated superior performance compared to traditional Backprojection and standard 2D Convolutional Decoders across all primary metrics. Quantitatively, the model achieved a PSNR of 26.32 dB and an SSIM of 61.38%, marking a substantial improvement over the Backprojection baseline (19.31 dB and 33.86%, respectively).

Quantitative comparison of the Fourier Feature model against Backprojection and Decoder 2D baselines.

Qualitatively, the Fourier-based approach was more effective at identifying vessel cross-sections and maintaining circular geometries. While the 2D Decoder often produced blurred artifacts or missed vessels entirely, the Fourier network preserved the structural integrity of the vessels more accurately. This suggests that transforming coordinates into sinusoids allows the model to "see" the sharp transitions of vessel walls more effectively.

Conclusion

This work demonstrates that 2D slices can be reconstructed from 1D rows, suggesting that complex 3D reconstruction can be decomposed into simpler, more computationally efficient 2D sub-problems. While the results are promising, limitations remain regarding the sharpness of fine vessels and the simplification of X-ray physics in the current simulation model. Future research will explore the integration of Physics-Informed Neural Networks (PINNs) and Diffusion Models to further enhance reconstruction fidelity and realism.

Summary updated

Abstract

<title>Abstract</title> 3D reconstruction of biplane cerebral angiograms remains a challenging, unsolved research problem due to the loss of depth information and the unknown pixelwise correlation between input images. The occlusions arising from only two views complicate the reconstruction of fine vessel details and the simultaneous addressing of inherent missing information. In this paper, we take an incremental step toward solving this problem by reconstructing the corresponding 2D slice of the cerebral angiogram using biplane 1D image data. We developed a coordinate-based neural network that encodes the 1D image data along with a deterministic Fourier feature mapping from a given input point, resulting in a slice reconstruction that is more spatially accurate. Using only one 1D row of biplane image data, our Fourier feature network reconstructed the corresponding volume slices with a peak signal-to-noise ratio (PSNR) of 26.32±0.36, a structural similarity index measure (SSIM) of 61.38±1.79, a mean squared error (MSE) of 0.0023±0.0002, and a mean absolute error (MAE) of 0.0364±0.0029. Our research has implications for future work aimed at improving backprojection-based reconstruction by first examining individual slices from 1D information as a prerequisite.
BibTeX
@inproceedings{wu2024fourier,
  author = {Sean M. Wu and Naoki Kaneko and David S. Liebeskind and Fabien Scalzo},
  title = {{Fourier Feature Network for 3D Vessel Reconstruction from Biplane Angiograms}},
  booktitle = {Research Square},
  year = {2024},
  doi = {10.21203/rs.3.rs-4103573/v1},
}
Figure