Abstract
While the computer vision industry is racing toward megapixel event cameras, this research reveals a surprising counter-trend: in low-light and high-speed conditions, lower-resolution sensors often outperform their high-resolution counterparts. By triggering events at much higher per-pixel rates, high-resolution sensors become hypersensitive to temporal noise and hardware bandwidth limits. This study suggests that "more pixels" is not a universal path to better performance in event-based vision.
Introduction
Event cameras are bio-inspired sensors that operate fundamentally differently from traditional frame-based cameras. Instead of capturing full images at fixed intervals, they report per-pixel intensity changes (events) asynchronously. This architecture offers significant advantages in dynamic range and latency. Following the trajectory of conventional imaging, manufacturers have rapidly increased resolutions from $128 \times 128$ to over $1280 \times 720$ (HD).
However, this "megapixel race" introduces a heavy price: a massive increase in data bandwidth and computational load. This paper investigates whether higher resolution actually yields better results for critical downstream tasks like image reconstruction, optical flow, and pose tracking, particularly in challenging environments like nighttime or high-speed motion.
The Resolution-Noise Trap
The researchers identify a critical trade-off between resolution and noise. In ideal lighting, high resolution provides the expected benefits of fine spatial detail. However, as pixel size decreases to achieve higher resolution, the per-pixel event rate increases. This happens because smaller pixels capture finer spatial gradients that trigger events more frequently during motion.
This increased event rate leads to the "Resolution-Noise Trap." Because high-resolution pixels trigger events faster, the time window between events ($\Delta t$) shrinks. When this window becomes smaller than the sensor's "cutoff frequency" (the physical speed limit of the pixel's circuitry), the sensor becomes highly susceptible to temporal noise. In low-light conditions, this manifests as a significant degradation of the signal-to-noise ratio, effectively creating "motion blur" within the event stream.
Theoretical Foundation: The Power Law of Detail
A core contribution of this work is the derivation of a theoretical relationship between pixel size and event generation. The researchers discovered that log brightness derivatives ($L_t$) follow a power law relative to pixel size ($p$).
As shown in the figure above, as pixel size $p$ decreases (moving left on the x-axis), the brightness derivative $L_t$ increases following a slope $\gamma^*$. This theoretical model explains why high-resolution sensors are inherently noisier: they are forced to operate at much higher temporal frequencies to capture the finer spatial details they are designed to resolve. If the hardware cannot support these frequencies—especially in the dark—performance collapses.
Experimental Results
The study utilized both synthetic and real-world multi-scale datasets to benchmark three baseline tasks: Event Integration (EI) for reconstruction, Event-based Photometric Flow (EPF), and Event-based Photometric Pose Tracker (EPPT).
Key Findings:
- Image Reconstruction: In nighttime settings, a $346 \times 346$ resolution outperformed a $1280 \times 1280$ resolution by nearly 0.5 dB in PSNR, despite having 13x fewer pixels. The high-res sensor's data was too noisy for effective reconstruction.
- Optical Flow: At high speeds in daytime, sensors with resolutions below $640 \times 640$ showed better performance than megapixel sensors. In nighttime conditions, the error for $1280$ resolution was often double that of the $640$ resolution.
- Pose Tracking: On nighttime datasets, the tracking error for $1280$ resolution jumped to 16.4 mm, while the $346$ resolution maintained a much lower error of 13.2 mm.
- Bandwidth Efficiency: Lower-resolution sensors (e.g., $346 \times 346$ or $640 \times 480$) often represent a "sweet spot," providing high performance with up to 8x less bandwidth than megapixel sensors.
Conclusion and Future Work
The megapixel race in event cameras is hitting a wall of physical and temporal constraints. While high resolution is beneficial in well-lit, slow-moving scenes, it becomes a liability in the high-speed or low-light scenarios where event cameras are supposed to excel.
Future research should focus on hardware-software co-design, such as sensors capable of adaptive resolution (dynamic pixel binning) to mitigate noise in challenging conditions, and tuning hardware contrast thresholds to balance fine detail against temporal noise. Ultimately, for many edge robotics applications, a medium-resolution sensor may be more robust and efficient than a megapixel one.
Summary updated