Abstract
Reconstructing high-quality video from high-speed motion is a significant challenge for standard cameras due to inherent motion blur. This paper introduces a novel, unsupervised optimization-based framework that fuses noisy data from event cameras with blurry frames from standard sensors. By modeling brightness changes using Bézier curves and incorporating per-event confidence weights, the proposed method achieves state-of-the-art reconstruction quality without the need for pre-training or massive datasets.
Introduction
Standard cameras capture light by integrating it over a fixed exposure time, which inevitably leads to motion blur when the scene contains fast-moving objects. In contrast, event cameras are bio-inspired sensors that asynchronously report per-pixel brightness changes with microsecond temporal resolution. While they excel at capturing motion, they do not record absolute brightness.
Existing attempts to merge these two modalities typically rely on deep neural networks. However, these "black box" models often suffer from domain gaps—performing well on synthetic data but failing on real-world footage—and struggle to distinguish between meaningful signal and the inherent noise in event streams.
Methodology
The authors propose a learning-free, optimization-based framework that treats video reconstruction as a continuous signal fitting problem. Instead of training a network, the system optimizes the parameters of a mathematical model to fit the observed data for every pixel.
Key Technical Innovations
- Bézier Curve Representation: To represent complex, smooth brightness transitions, the model utilizes piecewise Bézier functions. This provides greater flexibility than traditional linear assumptions.
- Per-Event Confidence Weights: To mitigate the impact of sensor noise, the model assigns a learnable "confidence weight" to each event. This allows the optimization process to ignore spurious noise that contradicts the overall brightness logic derived from the blurry frames.
- Integrated Brightness Model: The model represents the integral of brightness, allowing for an analytical calculation of expected long-exposure frames. This makes the optimization both more accurate and computationally efficient.
- Exposure-Based Control Points: Stability is maintained in regions with low motion by inserting control points at the start and end of camera exposures.
Evaluation and Results
The method was evaluated using the synthetic REDS Dataset and real-world data from a DAVIS 346C sensor.
Quantitative Performance
On the REDS dataset, the proposed method achieved a PSNR of 29.69 dB, outperforming the leading deep learning-based method (EVDI) by over 6 dB. It also reached a superior SSIM score of 0.9039, indicating high structural fidelity in the reconstructions.
Qualitative Performance
In real-world testing, the framework produced significantly sharper edges for fast-moving objects, such as spinning rackets or moving leaves. While deep learning baselines often produced "pulsing" artifacts or residual blur, this optimization-based approach maintained high-frequency details and temporal consistency.
Discussion and Future Work
This research demonstrates that a principled physical model can outperform complex deep learning architectures for specific sensor fusion tasks. The use of learnable confidence weights proves to be a robust mechanism for de-noising event streams.
However, the primary limitation is the computational cost. Because the system solves an optimization problem for every pixel, processing a short sequence can take several hours on high-end hardware. Future research will focus on accelerating this process, potentially through hybrid neural-optimization architectures, and extending the model to handle multi-channel color data more natively.
Summary updated