Traditional cameras require a straight, unobstructed path for photons to travel from a subject to the sensor. When an obstacle blocks this path, the scene is lost to view. However, light does not simply vanish when it hits a wall; it scatters. A matte white wall acts as a diffuse reflector, scattering incoming light in all directions. To the human eye, this scattered light looks like a flat, featureless color. But hidden within those subtle variations is a scrambled map of the hidden world.
Non-Line-of-Sight (NLOS) reconstruction is the computational framework used to treat ordinary surfaces—walls, floors, and corners—as accidental cameras, allowing us to mathematically unscramble this scattered light and see around obstructions.
Active NLOS: Time-of-Flight and Ellipsoidal Back-Projection
The most direct way to unscramble light is to control its emission. In active NLOS systems, a pulsed laser fires at a visible "relay surface" (such as a wall). The light bounces off the wall, hits the hidden object, bounces back to the wall, and is finally captured by an ultra-fast sensor capable of counting individual particles of light. As reviewed by Xue-mei Hu et al. in Emerging theories and technologies on computational imaging, this process relies on the precise timing of multi-bounce photons to map out hidden spaces.
Active Light Path:
Laser Source ---> Relay Wall ---> Hidden Object ---> Relay Wall ---> SPAD Detector
(Bounce 1) (Bounce 2) (Bounce 3)
Why does timing allow us to see shape? Because the speed of light is constant, the total time $t$ a photon takes to complete this three-bounce journey dictates the exact distance it traveled. If we know the position of the laser spot on the wall (the emitter focus) and the position of the wall spot being observed by the detector (the receiver focus), the hidden object must lie somewhere on a 3D surface where the sum of the distances to those two foci is constant.
In geometry, this surface is an ellipsoid. As Xiaochun Liu et al. demonstrate in Analysis of Feature Visibility in Non-Line-Of-Sight Measurements, this light transport can be modeled as an elliptical Radon transform. Every time-of-flight measurement represents an integral over an ellipsoidal shell in three-dimensional space. By scanning the laser across multiple points on the wall, the system generates thousands of overlapping ellipsoids. The intersection of these ellipsoids reveals the precise 3D geometry of the hidden scene.
Passive NLOS: Turning Corners into Apertures
Active systems require expensive, specialized hardware like femtosecond lasers and single-photon detectors. Passive NLOS instead leverages ambient light and existing architectural features to decode the hidden environment.
Consider a standard vertical wall corner. In Turning Corners into Cameras: Principles and Methods, Katherine L. Bouman et al. show that a sharp vertical edge functions as a natural, one-dimensional pinhole aperture.
Top-Down View of an Edge Camera:
Hidden Scene
( * ) Object A
\
\
--------\ Edge
\
\ Penumbra (Gradients on floor)
===========================================
Visible Floor
A completely open wall allows light from every angle of the hidden room to mix together on the floor, creating an uninterpretable blur. However, a corner blocks light from certain angles while letting light from other angles pass through. This creates a soft shadow gradient—a penumbra—on the floor.
Because the corner restricts the light paths, each point on the floor receives light from a slightly different angular wedge of the hidden scene. By taking the spatial derivative of the light intensity on the floor, the authors effectively isolate the light contributing from a single linear angle. This mathematical derivative acts as a computational lens, transforming subtle floor reflections into 1D videos of hidden moving objects.
When the intervening obstacle is not a clean corner but an unknown shape, the problem becomes more complex. Adam B. Yedidia et al. demonstrate in Using Unknown Occluders to Recover Hidden Scenes that this scenario can be framed as a 2D blind deconvolution task. The observed light pattern on a surface is the convolution of the hidden scene's radiance and the unknown occluder's shadow-casting geometry. By enforcing structural priors on what real-world scenes and objects look like, optimization algorithms can simultaneously solve for both the hidden scene's video and the shape of the obstacle casting the shadow.
Steady-State and Spatial Variations
If there are no sharp occluders and no ultra-fast lasers, can we still see around corners? The answer lies in analyzing spatial variations under different illumination patterns.
In Steady-State Non-Line-Of-Sight Imaging, Wenzheng Chen et al. demonstrate that standard continuous-wave light (like that from a consumer projector) and standard RGB cameras are sufficient for NLOS reconstruction. Instead of measuring the time it takes for light to return, the system changes where the light hits the relay wall.
Each position illuminated by the projector acts as a secondary light source that casts a unique distribution of light into the hidden room. The hidden object reflects this light back onto the wall, creating a distinct steady-state reflection pattern. Because different hidden geometries alter these spatial patterns in predictable ways, deep neural networks can be trained on synthetic light transport data to invert the mapping, recovering full-color images of hidden scenes without requiring picosecond timing.
Neural Fields and Alternative Modalities
The field has recently evolved by combining the physics of light transport with neural implicit representations. Rather than explicitly back-projecting voxels or solving matrix inversions, modern frameworks optimize a neural network to represent the hidden 3D volume.
For instance, Tzofi Klinghoffer et al. developed PlatoNeRF: 3D Reconstruction in Plato's Cave via Single-View Two-Bounce Lidar. By integrating two-bounce time-of-flight signals directly into the loss function of a Neural Radiance Field (NeRF), the authors show that a system can accurately supervise the density and color of occluded points. This allows for physically accurate, metric 3D reconstructions of hidden spaces from a single viewpoint without relying on generative hallucinations.
Visible Light vs. Radio Frequency Penetration:
Visible Light: Sensor ---> [ Opaque Wall ] x (Blocked)
Radio Frequency: Sensor ---> [ Opaque Wall ] ---> Hidden Object (Reconstructed)
Finally, researchers are expanding beyond the visible spectrum to solve the fundamental limitation of optical NLOS: the requirement of an open path around a corner or a clear reflection surface. When a target is completely enclosed or hidden behind a wall, visible light fails entirely due to material opacity.
To bypass this, Jiachen Lu et al. introduced GeRaF: Neural Geometry Reconstruction from Radio Frequency Signals. Radio frequency (RF) signals easily penetrate materials like drywall and cardboard but reflect off hidden targets. By redesigning volumetric rendering equations to account for the highly specular, lensless nature of radar waves, the authors demonstrate that neural implicit models can reconstruct millimeter-level 3D meshes of objects completely obscured from view, merging the physics of wave propagation with computational vision.
Key Foundational Work
Looking around the corner using transient imaging (2009) — contributed 4 concept elements to this direction.