Spatiotemporal-Aware Neural Fields for Dynamic CT Reconstruction

Qingyang Zhou, Yunfan Ye, Zhiping CaiAAAI 2025 · DOI

Introduction

Computed Tomography (CT) is a cornerstone of internal imaging, yet reconstructing dynamic scenes (4D-CT)—such as a breathing lung or deforming mechanical parts—remains a significant challenge. Traditional "gating" techniques often suffer from streak artifacts due to data sparsity. While Neural Radiance Fields (NeRF) have transformed 3D reconstruction, their application to CT is non-trivial. Unlike natural light, X-rays penetrate volumes, creating "high-rank" scenes that demand immense model capacity. Grid-based NeRF approaches often introduce "dot-like" artifacts and blurring at high resolutions caused by hash collisions in compressed memory grids.

Methodology: STNF4D

To address these limitations, we present STNF4D (SpatioTemporal-aware Neural Fields). The framework introduces two key innovations to handle high-resolution dynamic reconstruction efficiently.

4D Hash Grid Representation

The scene is decomposed into four orthogonal volumes: one purely spatial ($G_{XYZ}$) and three spatiotemporal ($G_{XYT}, G_{XZT}, G_{YZT}$). These are stored as compact hash grids, maintaining efficiency while capturing complex temporal dynamics.

Spatiotemporal Transformer (ST-Former)

The core of STNF4D is the ST-Former. Unlike standard methods that use Hadamard products (multiplication) to combine features—which can amplify noise from hash conflicts—STNF4D concatenates features and processes them through a self-attention mechanism. This allows the model to learn dependencies across grids and effectively suppress features corrupted by hash collisions.

Overview of the STNF4D pipeline, including the 4D hash grid representation and the ST-Former architecture.

Experimental Setup

The model was evaluated on two distinct types of data:

  • Medical Data: XCAT phantom and real patient data (TCIA) representing periodic breathing motion.
  • Industrial Data: Aluminum products under stress, featuring aperiodic motion and limited viewing angles.

Results and Discussion

STNF4D demonstrates substantial improvements over state-of-the-art methods like K-plane and Hex-plane.

  • Quantitative Performance: In medical scenarios, the model achieved a gain of 5.99 dB in PSNR. In industrial tasks, it outperformed the second-best method by 4.11 dB.
  • Qualitative Superiority: The framework effectively eliminates the "dot-like" artifacts common in high-resolution neural reconstructions.
  • Robustness: The model maintains high fidelity even under sparse-view sampling, a critical requirement for minimizing radiation exposure in clinical settings.
Qualitative comparison of reconstruction results with and without the ST-Former, showing significant reduction in artifacts.

Conclusion

STNF4D provides a robust solution for 4D-CT reconstruction by leveraging spatiotemporal-aware neural fields. By replacing feature multiplication with a Transformer-based attention mechanism, the model successfully mitigates hash collisions, enabling high-resolution imaging in both medical and industrial contexts. Future work will explore the integration of physical motion priors and scaling to ultra-high-resolution micro-CT.

Summary updated

BibTeX
@inproceedings{zhou2025spatiotemporal,
  author = {Qingyang Zhou and Yunfan Ye and Zhiping Cai},
  title = {{Spatiotemporal-Aware Neural Fields for Dynamic CT Reconstruction}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i10.33177},
}
Figure