Abstract
Event Cameras offer appealing advantages, including power efficiency and ultra-low latency, driving forward advancements in edge applications. In order to leverage mature frame-based algorithms, most approaches typically compute dense, image-like representations from sparse, asynchronous events. However, they are often unable to capture comprehensive information or are computationally intensive, which hinders the edge deployment of event-based vision. Meanwhile, pillar-based paradigms have been proven to be efficient and well established for dense representations of sparse data. Hence, from a novel pillar-based perspective, we present EventPillars, an efficient, comprehensive framework for dense event representations. To summarize, it (i) incorporates the Temporal Event Range to describe an intact temporal distribution, (ii) Activates the Event Polarities to explicitly record the scene dynamics, (iii) enhances the target awareness by a spatial attention prior from Normalized Event Density, (iv) can be plug-and-played into different downstream tasks. Extensive experiments show that our EventPillars records a new state-of-the-art precision on object recognition and detection datasets with surprisingly 9.2× and 4.5× lower computation and storage consumption. This brings a new insight into dense event representations and is promising to boost the edge deployment of event-based vision.
Introduction
Event cameras represent a paradigm shift in visual sensing. Unlike traditional cameras that capture frames at fixed intervals, event cameras report pixel-wise brightness changes asynchronously. This results in a sparse, irregular stream of $(x, y, t, p)$ coordinates, offering ultra-low latency and high dynamic range. However, standard computer vision architectures like CNNs require dense, grid-like inputs to process this data effectively.
Existing methods to bridge this gap often fall into two extremes: simple representations like Event Frames that discard critical temporal and polarity information, or heavy representations like Voxel Grids or Event Spike Tensors (EST) that are computationally prohibitive for edge devices. EventPillars addresses this "Goldilocks" problem by introducing a pillar-based representation that is both comprehensive and highly efficient.
The EventPillars Framework
Inspired by PointPillars in LiDAR processing, EventPillars collapses the sparse event stream into a dense 4-channel tensor ($4 \times H \times W$). This representation is built from three core components designed to capture the essence of event data:
- Temporal Event Range (TER): Instead of only recording the latest timestamp, TER captures both the Newest (NET) and Earliest (EET) timestamps within a window. This allows the model to perceive the temporal span and trajectory of motion.
- Activated Event Polarity (AEP): To preserve the direction of brightness changes without losing information to cancellation, AEP uses a continuous activation function to summarize polarity balance while filtering noise.
- Normalized Event Density (NED): Serving as a spatial attention prior, NED normalizes event counts across the frame. This helps the model distinguish high-density targets from background noise, ensuring robustness across different sensor types and object speeds.
Experimental Evaluation
The authors evaluated EventPillars on five major benchmarks, including N-ImageNet for recognition and Gen1 for detection. The results demonstrate a superior balance between performance and efficiency.
EventPillars achieved state-of-the-art accuracy, reaching 97.1% on N-Cars and 85.3% on N-Caltech101. In object detection, it set a new record on the Gen1 dataset with 53.1% mAP. Crucially, these gains come with a massive reduction in overhead: EventPillars is 9.2× faster than EST on a standard CPU (0.62ms vs 5.73ms) and requires 4.5× less storage.
Key Insights and Conclusion
A critical finding of this work is the value of the "Earliest" timestamp (EET). Ablation studies showed that including EET alongside NET boosts accuracy significantly by providing a sense of motion duration. Furthermore, the use of relative density (NED) makes the representation invariant to sampling rates, enhancing generalization across different hardware.
While EventPillars currently uses fixed temporal windows, future work may explore multi-scale temporal partitioning to capture more complex dynamics. By slashing computational costs while improving accuracy, EventPillars provides a promising path for deploying high-performance event-based vision on resource-constrained edge devices.
Summary updated