Introduction
Conventional digital cameras often struggle with high-speed motion, as they average light over an exposure window, resulting in motion blur. To overcome this, "spike cameras" inspired by the human fovea have been developed. These sensors fire asynchronous binary spikes whenever a pixel accumulates a specific photon threshold, enabling extremely high temporal resolution (tens of thousands of Hz).
However, spike cameras currently suffer from limited spatial resolution. Traditional reconstruction methods typically aim for the sensor's native resolution. Simply applying standard Super-Resolution (SR) techniques to reconstructed low-resolution images often fails because fine details are lost during the initial reconstruction phase. This paper introduces the Motion-Guided Spike Camera Super-resolution (MGSR) framework, the first to perform SR directly from raw spike data.
Methodology
The MGSR framework treats image reconstruction as an optimization problem rather than a simple averaging process. The core innovation is a mathematical formulation that links the light intensity of a scene point to the spikes fired by a sensor pixel.
By accounting for relative motion between the scene and the camera, the framework uses motion as a spatial sampling advantage. A single scene point is captured by different pixels over time, providing sub-pixel information. The pipeline involves light inference for initial low-res images, motion estimation via optical flow, and an optimization step using gradient descent to reconstruct the high-resolution intensity image.
Experimental Results
The researchers evaluated MGSR using the PKU-Spike-High-Speed Dataset and new sequences captured with an FSM spike camera (400x250 resolution at 20,000–40,000 Hz). The scenes included both object motion (e.g., a falling doll, high-speed cars and trains) and camera ego-motion.
Qualitatively, MGSR outperforms existing methods like TFP, TFI, and TVS by recovering sharp textures and legible details that others miss. Quantitatively, MGSR consistently achieved the best (lowest) scores in no-reference quality metrics such as BRISQUE and NIQE. For instance, in a 350 km/h "Train" sequence, MGSR scored 3.14 on NIQE, significantly better than TFP (10.62) and TFI (6.49).
Discussion and Conclusion
The key insight of this work is that motion, often viewed as a source of blur, can be leveraged as a "scanning" mechanism for sub-pixel sampling. The paper demonstrates that "direct" SR from spikes is superior to "pipelined" approaches because it preserves the precise timing of photons, which contains high-frequency spatial information.
While highly effective, the method's performance depends on the accuracy of optical flow estimation. Future work may involve integrating deep learning-based motion estimation or end-to-end neural networks to enhance robustness and computational efficiency.
Summary updated