On a Hidden Property in Computational Imaging

Yinan Feng, Yinpeng Chen, Yueh Lee, Youzuo Lin

Abstract

Computational imaging plays a vital role in various scientific and medical applications, such as Full Waveform Inversion (FWI), Computed Tomography (CT), and Electromagnetic (EM) inversion. These methods address inverse problems by reconstructing physical properties (e.g., the acoustic velocity map in FWI) from measurement data (e.g., seismic waveform data in FWI), where both modalities are governed by complex mathematical equations. In this paper, we empirically demonstrate that despite their differing governing equations, three inverse problems (FWI, CT, and EM inversion) share a hidden property within their latent spaces. Specifically, using FWI as an example, we show that both modalities (the velocity map and seismic waveform data) follow the same set of one-way wave equations in the latent space, yet have distinct initial conditions that are linearly correlated. This suggests that after projection into the latent embedding space, the two modalities correspond to different solutions of the same equation, connected through their initial conditions. Our experiments confirm that this hidden property is consistent across all three imaging problems, providing a novel perspective for understanding these computational imaging tasks.

Introduction

Computational imaging is central to modern scientific discovery and medical diagnostics, enabling the reconstruction of internal physical properties from external measurements. Tasks such as Full Waveform Inversion (FWI) in geophysics, Computed Tomography (CT) in medicine, and Electromagnetic (EM) inversion are fundamentally inverse problems. Traditionally, these problems are solved by modeling the physical laws—such as the acoustic wave equation or Maxwell’s equations—that govern the relationship between measurements and properties.

While deep learning has significantly advanced the state-of-the-art by treating these as image-to-image translation tasks, the resulting models often operate as "black boxes." This lack of interpretability obscures the underlying mathematical relationships that might exist within the learned latent spaces. This research investigates whether an elegant, shared mathematical structure exists in the latent space across diverse computational imaging modalities, potentially mirroring the physical laws of the original space.

The HINT Framework

The authors introduce the HIddeN properTy (HINT) framework, which posits that measurement data ($P$) and target physical properties ($\psi$) share a common mathematical foundation in their latent representations. Specifically, it is hypothesized that both modalities satisfy the same set of one-way wave equations in the latent space, differing only in their initial conditions.

The HINT framework illustrates the relationship between physical space and latent space across different modalities.

The HINT framework utilizes a technique called FINOLA (First-Order Norm+Linear Autoregressive process) to implement these latent dynamics. The process involves:

  1. Encoding: Measurement data is projected into a latent vector ($v_P$).
  2. Linear Mapping: A simple linear transformation ($T$) maps $v_P$ to the latent vector for the physical property ($v_\psi$).
  3. Shared Propagation: Both vectors serve as initial conditions for a shared "multi-path FINOLA" layer, which evolves the latent states according to a shared set of wave equations with wave speeds $\Lambda$.
  4. Decoding: Task-specific decoders reconstruct the final outputs from these evolved feature maps.

Experimental Setup

The universality of the HINT property was tested across three distinct domains:

  • Seismic Imaging (OpenFWI): A comprehensive collection of ten datasets representing various subsurface structures (e.g., FlatVel, CurveFault).
  • Medical Imaging (RSNA/ASNR CT): A large-scale dataset of approximately 53,000 brain CT scans.
  • Electromagnetic Inversion (Kimberlina-Reservoir): Focused on subsurface conductivity reconstruction.

Results and Analysis

The empirical results strongly support the existence of the shared latent property.

Shared Latent Dynamics

A key experiment compared the performance of models using a single set of shared wave equations (Shared FINOLA) against those using separate equations for each modality. As shown in the following comparison, the performance in terms of Structural Similarity Index (SSIM) remained nearly identical across all OpenFWI datasets. This confirms that the latent dynamics of measurement data and physical properties are indeed governed by the same underlying "physics" in the embedding space.

SSIM comparison across OpenFWI datasets demonstrating that shared latent wave equations maintain performance levels.

Linearity in Latent Space

Further analysis investigated the nature of the mapping between the latent initial conditions. Replacing the simple linear converter with more complex non-linear architectures, such as Multi-Layer Perceptrons (MLPs) or Maxout networks, yielded no significant performance gains. This suggests that the complex, non-linear relationship between measurements and properties in the physical world simplifies into a linear correlation within the latent space.

Conclusion

This study reveals a fundamental "hidden property" in computational imaging: despite disparate physical governing equations, different modalities share a common set of one-way wave equations in their latent spaces. By imposing this mathematical structure as a hard constraint, the HINT framework achieves competitive performance while enhancing model interpretability.

Future work will focus on improving the capture of fine structural details in CT imaging, where HINT currently produces smoother results than traditional methods. Additionally, addressing dataset interference in multi-task training remains an open challenge. Ultimately, these findings pave the way for more efficient, physics-informed latent space solvers in computational imaging.

Summary updated

Abstract

Computational imaging plays a vital role in various scientific and medical applications, such as Full Waveform Inversion (FWI), Computed Tomography (CT), and Electromagnetic (EM) inversion. These methods address inverse problems by reconstructing physical properties (e.g., the acoustic velocity map in FWI) from measurement data (e.g., seismic waveform data in FWI), where both modalities are governed by complex mathematical equations. In this paper, we empirically demonstrate that despite their differing governing equations, three inverse problems (FWI, CT, and EM inversion) share a hidden property within their latent spaces. Specifically, using FWI as an example, we show that both modalities (the velocity map and seismic waveform data) follow the same set of one-way wave equations in the latent space, yet have distinct initial conditions that are linearly correlated. This suggests that after projection into the latent embedding space, the two modalities correspond to different solutions of the same equation, connected through their initial conditions. Our experiments confirm that this hidden property is consistent across all three imaging problems, providing a novel perspective for understanding these computational imaging tasks.
BibTeX
@misc{feng2024hidden,
  author = {Yinan Feng and Yinpeng Chen and Yueh Lee and Youzuo Lin},
  title = {{On a Hidden Property in Computational Imaging}},
  year = {2024},
  eprint = {2410.08498},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.08498},
}
Figure