Abstract
Non-line-of-sight (NLOS) imaging, recovering the hidden volume from indirect reflections, has attracted increasing attention due to its potential applications. Despite promising results, existing NLOS reconstruction approaches are constrained by the reliance on empirical physical priors, e.g., single fixed path compensation. Moreover, these approaches still possess limited generalization ability, particularly when dealing with scenes at a low signal-to-noise ratio (SNR). To overcome the above problems, we introduce a novel learning-based solution, comprising two key designs: Learnable Path Compensation (LPC) and Adaptive Phasor Field (APF). The LPC applies tailored path compensation coefficients to adapt to different objects in the scene, effectively reducing light wave attenuation, especially in distant regions. Meanwhile, the APF learns the precise Gaussian window of the illumination function for the phasor field, dynamically selecting the relevant spectrum band of the transient measurement. Experimental validations demonstrate that our proposed approach, only trained on synthetic data, exhibits the capability to seamlessly generalize across various real-world datasets captured by different imaging systems and characterized by low SNRs.
Introduction
Non-Line-of-Sight (NLOS) imaging is often described as the "superpower" of seeing around corners. By analyzing light that has bounced off a visible "relay wall" to illuminate hidden objects, researchers can reconstruct 3D volumes of scenes not directly visible to a camera. This technology holds immense potential for autonomous driving, search-and-rescue operations, and industrial inspection.
However, current NLOS methods face significant real-world challenges. First, Radiometric Intensity Fall-off (RIF) causes light to weaken significantly as it travels. Traditional algorithms attempt to counteract this using fixed compensation factors (e.g., $1/r^2$ or $1/r^4$). Unfortunately, real-world scenes often contain a mix of materials—such as diffuse surfaces and retro-reflective objects—that attenuate light at different rates. A uniform compensation factor often fails to reconstruct complex scenes accurately. Second, the Signal-to-Noise Ratio (SNR) in NLOS is typically very low. High-quality data often requires long acquisition times, which is impractical for dynamic scenarios. Existing models struggle to generalize to the noisy conditions found in short-exposure, real-world captures.
Methodology: Learnable Physical Priors
To address these limitations, this work introduces a hybrid architecture that integrates learnable physical priors into a wave-based propagation pipeline. By replacing empirical parameters with neural modules, the framework achieves high flexibility without losing the grounding of physical laws.
Learnable Path Compensation (LPC)
Instead of applying a single, global compensation coefficient, the LPC module utilizes a 3D Convolutional Neural Network (CNN) to predict spatial probabilities for various compensation weights. This allows the model to adaptively "boost" the signal for distant or low-reflectivity objects while maintaining the integrity of closer, brighter objects. This material-aware approach is crucial for scenes containing diverse objects like a diffuse chair and a retro-reflective deer.
Adaptive Phasor Field (APF)
The APF module operates in the frequency domain to improve noise resilience. It learns to predict the optimal width (standard deviation) of a Gaussian window for the illumination function. This acts as a dynamic, smart band-pass filter that selects relevant signal frequencies while suppressing high-frequency noise. By tailoring the filter to the specific SNR of the input measurement, the model can "squint" through noise that would otherwise render the reconstruction unrecognizable.
Experimental Evaluation
The model was trained exclusively on a synthetic dataset of 2,704 samples. Despite this synthetic-only training, the model demonstrates remarkable generalization to real-world data.
Datasets and Robustness
The authors evaluated the framework on several challenging benchmarks:
- Public Real-World Data: Testing on datasets like FK and NLOST, specifically focusing on low-acquisition-time versions (e.g., 10 minutes) to simulate high-noise environments.
- Self-Captured Data: Using a custom confocal NLOS system (532nm laser and SPAD detector), the authors captured complex scenes featuring diverse materials to test the LPC module's effectiveness.
Key Results
The proposed method consistently outperformed both traditional physics-based algorithms and state-of-the-art (SOTA) learning models. Quantitatively, it achieved a 23.99 dB PSNR on synthetic test sets, surpassing traditional RSD (21.74 dB) and the Transformer-based NLOST (23.74 dB). Qualitatively, the model maintains structural integrity even at extreme noise levels (3 dB SNR), where other methods produce unrecognizable results. Furthermore, the architecture is efficient, requiring roughly half the GPU memory of competing SOTA models.
Conclusion
This research demonstrates that physics-informed learning is a powerful paradigm for NLOS imaging. By embedding the structure of path compensation and phasor field filtering into a neural network, the model achieves superior generalization and robustness compared to "black box" approaches. Future work will focus on bridging the remaining domain gap in sensor modeling and extending the framework to non-confocal imaging systems.
Summary updated