On a Hidden Property in Computational Imaging

Yinan Feng, Yinpeng Chen, Yueh Z Lee, Youzuo LinSubmission to ICLR 2025 · Rejected

TL;DR

Deep learning models for computational imaging often treat inverse problems as black-box image-to-image translation tasks. This paper reveals a striking hidden mathematical structure shared across distinct inverse problems: in the latent space, both the raw sensor measurements and the underlying physical properties satisfy the exact same set of one-way wave equations, differing only by linearly correlated initial conditions.

What Problem Does This Paper Address?

Computational imaging solves inverse problems—reconstructing internal physical properties (like seismic velocity maps, internal tissue attenuation, or electrical conductivity) from indirect sensor measurements (seismic waves, X-ray projections, or electromagnetic fields). In the physical world, these modalities are governed by vastly different partial differential equations (PDEs), such as the acoustic wave equation, the Radon transform, or Maxwell's equations.

While deep learning architectures (like convolutional or Transformer encoder-decoders) have shown impressive empirical performance on these tasks, their latent bottleneck representations remain largely uninterpretable. The authors ask: Does a unifying mathematical relationship exist between measurements and targets within the latent space?

Methods

The authors introduce HINT (HIddeN properTy), a unified framework extending the single-image autoregressive wave formulation known as FINOLA (First-Order Norm+Linear Autoregression):

  1. Latent Encoding: A measurement $P$ (e.g., raw waveform) is encoded via a Transformer into a compact latent vector $v_P$.
  2. Linear Latent Bridge: The latent representation of the physical property $v_\psi$ is derived from $v_P$ via a simple linear projection matrix $T$ ($v_\psi = T v_P$).
  3. Shared Latent Wave Propagation: Both $v_P$ and $v_\psi$ serve as initial boundary conditions for the exact same multi-path FINOLA autoregression layer. This process is governed by a shared system of one-way wave equations: $$\frac{\partial \zeta}{\partial x} = \Lambda \frac{\partial \zeta}{\partial y}$$ where $\Lambda$ represents identical, learnable wave speeds shared by both modalities.

  4. Reconstruction & Prediction: The resulting feature maps $z_P$ and $z_\psi$ are decoded by lightweight convolutional decoders: $D_P$ reconstructs the input measurements, and $D_\psi$ inverts the target physical property. The system is trained end-to-end with combined $L_1$ and $L_2$ losses.

Datasets

The authors evaluate HINT across three distinct physical domains:

  • Full Waveform Inversion (FWI): OpenFWI benchmark, evaluating on 10 datasets across three families (FlatVel-A/B, CurveVel-A/B, FlatFault-A/B, CurveFault-A/B, and Style-A/B).
  • Computed Tomography (CT): Brain CT scans from the RSNA 2019 Intracranial Hemorrhage Challenge (47,000 train / 6,000 test images), paired with simulated projections mimicking a stationary head CT (s-HCT) geometry with sparse, asymmetric carbon nanotube X-ray source arrays.
  • Electromagnetic (EM) Inversion: The Kimberlina-Reservoir dataset (780 simulated samples; 750 train / 30 test) modeling subsurface conductivity for $\text{CO}_2$ storage monitoring.

Key Insights and Ideas

  • A Dual World Governed by One Wave Equation: Despite stark physical differences between seismic sound propagation, CT ray attenuation, and electromagnetic field equations, in the latent space both modalities behave as solutions to a single, shared system of one-way wave equations with identical propagation velocities ($\Lambda$).
  • Linearity Suffices: The initial conditions of these latent wave equations are related purely linearly. Experiments replacing the linear layer $T$ with nonlinear converters (two-layer MLPs or two-piece Maxout networks) yielded no performance gains, proving that the cross-modal connection in latent space is fundamentally linear.
  • Cross-Dataset Latent Universality: The same wave speeds can be shared not only across different modalities within an inverse task, but across completely distinct structural geologies within multi-dataset benchmarks.

Key Results

  • FWI: When jointly trained across all 10 OpenFWI datasets, HINT matches or slightly outperforms BigFWI while using 25% fewer parameters (18.2M vs. 24.4M parameters tied to inversion). It reliably achieves measurement reconstruction SSIM up to 0.9998 while outperforming Auto-Linear across MAE, MSE, and SSIM.
  • CT: HINT outperforms InversionNet across all metrics (MAE: 31.95 vs. 63.27; MSE: 9,754.48 vs. 274,350.78; SSIM: 0.9843 vs. 0.9684). While classical iterative reconstruction (SIRT) achieves lower MSE (6,510.67), HINT achieves better MAE (31.95 vs. 45.67) and avoids severe scan artifacts.
  • EM Inversion: HINT reduces reconstruction errors substantially compared to Auto-Linear and InversionNet (MAE of 0.0018 vs. 0.0044 and 0.0133; MSE of 3.34e-5 vs. 1.92e-4 and 8.55e-4; SSIM of 0.9937 vs. 0.9700 and 0.9175).
  • Resolution Invariance: Ablations across feature map resolutions from $5 \times 5$ to $35 \times 35$ confirm the stability of the latent wave dynamic across varying propagation steps.

Limitations and Future Work

  • Cross-Dataset Interference: When jointly trained across diverse geological datasets, visual artifacts occasionally bleed over from one dataset style into another (e.g., patterns from other sets leaking into CurveFault-B).
  • Over-Smoothing in CT: In CT reconstructions, HINT produces clean, artifact-free outputs but tends to over-smooth subtle edges and fine anatomical details compared to classical iterative solvers like SIRT.
  • Synthetic Domain Constraint: Evaluations were performed on simulated/synthetic benchmarks; validation on complex, real-world field acquisition data remains to be demonstrated.

Summary updated

Abstract

Computational imaging plays a vital role in various scientific and medical applications, such as Full Waveform Inversion (FWI), Computed Tomography (CT), and Electromagnetic (EM) inversion. These methods address inverse problems by reconstructing physical properties (e.g., the acoustic velocity map in FWI) from measurement data (e.g., seismic waveform data in FWI), where both modalities are governed by complex mathematical equations. In this paper, we empirically demonstrate that despite their differing governing equations, three inverse problems—FWI, CT, and EM inversion—share a hidden property within their latent spaces. Specifically, using FWI as an example, we show that both modalities (the velocity map and seismic waveform data) follow the same set of one-way wave equations in the latent space, yet have distinct initial conditions that are linearly correlated. This suggests that after projection into the latent embedding space, the two modalities correspond to different solutions of the same equation, connected through their initial conditions. Our experiments confirm that this hidden property is consistent across all three imaging problems, providing a novel perspective for understanding these computational imaging tasks.
BibTeX
@misc{feng2024hidden,
  author = {Yinan Feng and Yinpeng Chen and Yueh Z Lee and Youzuo Lin},
  title = {{On a Hidden Property in Computational Imaging}},
  year = {2024},
  eprint = {2410.08498},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.08498},
}

Loading related content...

Figure