Event Cameras Meet SPADs for High-Speed, Low-Bandwidth Imaging

Manasi Muglikar, Siddharth Somasundaram, Akshat Dave, Edoardo Charbon, Ramesh Raskar, Davide Scaramuzza

Introduction

Capturing high-speed motion in low-light environments remains a significant hurdle in computer vision. Conventional CMOS sensors face a fundamental trade-off: long exposure times gather sufficient light but introduce severe motion blur, while short exposures minimize blur at the cost of high noise levels. While specialized sensors like Single-Photon Avalanche Diodes (SPADs) and Event Cameras offer potential solutions, they introduce their own challenges—primarily a massive data bandwidth requirement for SPADs and a lack of absolute intensity information for event cameras.

Figure 1: Comparison of imaging technologies across the performance-bandwidth trade-off. The proposed fusion method (Ours) approaches the "Ideal" zone by achieving high performance in low-light, dynamic scenes with significantly reduced bandwidth.

This research introduces a novel sensor fusion framework that combines the microsecond precision of SPADs with the low-bandwidth efficiency of event cameras. By using events to "guide" and deblur SPAD data, the authors achieve high-quality, 100 kHz image reconstruction in near-darkness while reducing data bandwidth by up to 1000x compared to existing methods.

Methodology

The proposed framework addresses the bandwidth-performance trade-off through three core components:

  1. Nonlinear Event Double Integration (NEDI): Unlike traditional linear deblurring, NEDI accounts for the unique non-linear response function of SPAD sensors. It uses high-temporal-resolution event data to sharpen aggregated, blurry SPAD frames.
  2. Asynchronous Kalman Filter (AKF): This filter treats events and deblurred SPAD frames as asynchronous updates to reconstruct a continuous video stream. It incorporates sophisticated noise models, including shot noise and sensor refractory periods.
  3. Adaptive SPAD Sampling: To optimize bandwidth, the system triggers SPAD frames based on the uncertainty estimate from the Kalman filter. When the event camera provides sufficient information, the SPAD remains inactive, saving power and data.

Experimental Evaluation

The approach was validated using SimSPAD, a high-fidelity synthetic dataset, and HS-ESPAD, a real-world hardware rig featuring a SwissSPAD2 sensor and a Prophesee Gen4 event camera. Testing included challenging scenarios such as rapidly rotating fans and high-contrast targets in low-light settings.

Figure 6: Comparison of sensor properties and visual performance on a rotating fan. The proposed method balances the high-speed performance of event cameras with the low-light sensitivity of SPADs while maintaining manageable data bandwidth.

Key Results and Insights

The fusion of these sensors leverages their complementary physics: SPADs provide absolute intensity ("what"), while event cameras provide precise motion timing ("where/when"). Key achievements include:

  • Enhanced SNR: A 5 dB PSNR improvement in low-light reconstruction at 100 kHz over conventional cameras.
  • Extreme Efficiency: A 1000x reduction in bandwidth and power requirements compared to SPAD-only baselines like Quanta Burst Photography.
  • HDR Capability: Successful reconstruction of details in both dark indoor and bright outdoor regions simultaneously.

Conclusion and Future Work

While the current system demonstrates a breakthrough in high-speed, low-light imaging, challenges remain in sensor calibration and hardware integration. Future work aims to develop a single hybrid chip to perform this fusion at the hardware level, further pushing the boundaries of imaging in the dark.

Summary updated

BibTeX
@article{muglikar2026event,
  author = {Manasi Muglikar and Siddharth Somasundaram and Akshat Dave and Edoardo Charbon and Ramesh Raskar and Davide Scaramuzza},
  title = {{Event Cameras Meet SPADs for High-Speed, Low-Bandwidth Imaging}},
  journal = {IEEE TPAMI},
  year = {2026},
  url = {https://openreview.net/forum?id=u3gLz8r9De},
}
Figure