At the heart of low-light and high-speed imaging lies a fundamental physical conflict. Light collection is inherently a counting problem: photons arrive at a camera sensor according to a Poisson process. To produce a clear image with a high Signal-to-Noise Ratio (SNR), the sensor must collect a sufficient number of photons.
In dim environments, an imaging system has two primary levers to increase its photon count: widen the aperture or lengthen the exposure time. Widening the aperture reduces the depth of field, which can throw crucial scene elements out of focus. Lengthening the exposure time introduces a severe penalty: if the camera shakes or objects move during the window when the shutter is open, the arriving photons are smeared across multiple pixels, causing motion blur.
To freeze rapid motion, the exposure time must be short. However, a short exposure limits photon intake, forcing the system to increase its analog gain (ISO). This amplifies both the inherent photon arrival variance (shot noise) and the sensor's electronic floor (read noise), resulting in an image dominated by extreme noise.
This exposure-noise-blur trade-off represents a hard physical constraint of traditional frame-based sensors. Over the years, researchers have developed distinct computational and hardware strategies to break this trilemma, moving from intelligent multi-frame fusion to hybrid sensor architectures and single-photon imaging.
Splitting the Exposure: Burst and Bracketing Fusion
Instead of relying on a single long exposure, modern computational photography splits the light collection process across time or space, using deep learning to recombine the pieces.
Burst Photography
One straightforward approach is to capture a rapid succession of short-exposure frames, a technique known as burst photography (Godard et al.). Each individual frame is captured quickly enough to freeze motion, meaning it remains sharp, but it suffers from low SNR.
Mathematically, if noise is zero-mean and random, averaging $N$ perfectly aligned frames reduces the noise variance by a factor of $\sqrt{N}$ while preserving the underlying signal. The core challenge is non-rigid alignment: if the scene or camera moves between frames, simple averaging creates a ghosted or blurry result. Modern architectures handle this by predicting adaptive, localized alignment kernels (Marinc et al.) or using raw sensor data pipelines (Chen et al.) to robustly merge features without introducing motion artifacts.
Exposure Bracketing and Dual-Exposure Patterns
Rather than capturing uniform short exposures, exposure bracketing mixes short-exposure and long-exposure frames (Goyal et al.; Zhang et al.). This creates a complementary relationship:
- The long exposure provides clean low-frequency illumination data (low noise) but lacks sharp high-frequency details due to motion blur.
- The short exposure provides sharp high-frequency structural boundaries (no blur) but is plagued by high-frequency noise.
Deep neural networks can exploit this complementarity, using the sharp edges of the short exposure as a structural blueprint to guide the deblurring of the long exposure, while using the long exposure to denoise the short one.
To eliminate the temporal "ghosting" that occurs when these exposures are taken sequentially, the authors of Modeling Dual-Exposure Quad-Bayer Patterns for Joint Denoising and Deblurring (Zhao et al.) designed a sensor-level solution. By interleaving short- and long-exposure pixels simultaneously within a single Quad-Bayer mosaic pattern, they capture both modalities at the exact same moment, bypassing temporal misalignment entirely.
Un-Integrating Blur with Event Cameras
A traditional frame-based camera acts as a temporal integrator. Over an exposure window $T$, each pixel sums up all incoming light to produce an intensity value:
$$I_{\text{blur}} = \int_0^T L(t) \, dt$$
If the latent image $L(t)$ changes during this interval, the integration process destroys the temporal trajectory of the scene, outputting a blurred image.
Event cameras operate under a completely different paradigm. Instead of capturing absolute intensity at fixed intervals, each pixel operates independently and asynchronously, emitting a binary "event" the moment the local log-intensity changes by a set threshold:
$$\Delta \ln L(t) = \pm C$$
Because event cameras sample at microsecond-level resolution, they are virtually immune to motion blur and possess an exceptionally high dynamic range. However, they only record changes in light, meaning they lack static texture, color, and absolute intensity information (Ercan et al.).
The trade-off can be resolved by pairing these two sensors. The blurry frame provides the total integrated intensity (the area under the curve), while the event stream provides the exact relative path (the derivative) that the light took during that integration window.
Using the Event-based Double Integral (EDI) framework introduced by Pan et al., or continuous intensity recovery models (Song et al.), algorithms can mathematically invert the integration process. By using the high-speed event data to "un-integrate" the blurry frame, researchers can reconstruct crystal-clear, high-frame-rate video from a single blurry, low-light capture (Jeong et al.; Liang et al.).
Eliminating Read Noise via Single-Photon Imaging
If splitting an exposure into multiple frames solves motion blur, why not simply run a standard CMOS sensor at 100,000 frames per second? The barrier is read noise.
Every time a traditional CMOS pixel reads out its accumulated charge, electronic components introduce a small, fixed amount of noise. If you split a 1-second exposure into 100,000 separate frames, you incur that read noise penalty 100,000 times. In low-light conditions, this noise floor completely overwhelms the sparse photons arriving at the sensor.
Single-Photon Avalanche Diodes (SPADs) bypass this limitation. SPADs are digital, binary sensors that register the arrival of individual photons with sub-nanosecond precision. In Quanta Burst Photography, a method developed by Ma, the sensor records a massive stream of 1-bit frames at extreme speeds. Because SPAD readouts are digital counts rather than analog voltage measurements, they feature zero read noise.
As a result, reading out 100,000 1-bit frames and summing them up yields the exact same noise profile as a single long exposure. Crucially, because these 1-bit snapshots are captured at a quantum timescale before motion blur can manifest, algorithms can track, align, and re-sample the photon streams prior to merging them (Jungerman; Mos et al.). This completely decouples exposure time from motion blur, enabling clear imaging in conditions of extreme motion and ultra-low light.