Abstract
Event-based vision revolutionizes traditional image sensing by capturing asynchronous intensity variations rather than static frames, enabling ultrafast temporal resolution, sparse data encoding, and enhanced motion perception. While this paradigm offers significant advantages, conventional event-based datasets impose a fixed thresholding constraint to determine pixel activations, severely limiting adaptability to real-world environmental fluctuations. Lower thresholds retain finer details but introduce pervasive noise, whereas higher thresholds suppress extraneous activations at the expense of crucial object information. To mitigate these constraints, we introduce the Event-Based Crossing Dataset (EBCD), a comprehensive dataset tailored for pedestrian and vehicle detection in dynamic outdoor environments, incorporating a multi-thresholding framework to refine event representations. By capturing event-based images at ten distinct threshold levels (4, 8, 12, 16, 20, 30, 40, 50, 60, and 75), this dataset facilitates an extensive assessment of object detection performance under varying conditions of sparsity and noise suppression. We benchmark state-of-the-art detection architectures—including YOLOv4, YOLOv7, EfficientDet-b0, MobileNet-v1, and Histogram of Oriented Gradients (HOG)—to experiment upon the nuanced impact of threshold selection on detection performance. By offering a systematic approach to threshold variation, we foresee that EBCD fosters a more adaptive evaluation of event-based object detection, aligning diverse neuromorphic vision with real-world scene dynamics.
Introduction
Traditional frame-based cameras capture the world in static intervals, often missing high-speed dynamics or struggling with high-contrast scenes. Event cameras, or neuromorphic sensors, address these limitations by recording only changes in pixel intensity. However, a critical bottleneck in their deployment is the activation threshold ($T_h$). A fixed threshold often fails in dynamic outdoor environments: low thresholds capture essential motion but are overwhelmed by noise (e.g., wind, sensor jitter), while high thresholds may filter out significant signals like distant pedestrians. The Event-Based Crossing Dataset (EBCD) addresses this by providing a multi-threshold benchmark for optimizing object detection in real-world scenarios.
Methodology
The researchers developed a multi-thresholding framework to explore the sensitivity-noise trade-off. Using 33 crossing videos from the NTU Pedestrian Dataset, they generated event-based representations across ten distinct threshold levels ($T_h \in \{4, 8, 12, 16, 20, 30, 40, 50, 60, 75\}$).
To quantify the impact of noise, the study introduced an algorithm to calculate the ratio of "useful" pixels (within object bounding boxes) to "noise" pixels (background). This data was then used to benchmark five architectures: YOLOv4, YOLOv7, EfficientDet-b0, MobileNet-v1, and Histogram of Oriented Gradients (HOG).
The Event-Based Crossing Dataset (EBCD)
The EBCD consists of approximately 30,000 multi-threshold images derived from ~3,000 annotated frames. It categorizes objects into two primary classes: Pedestrians and Vehicles.
The dataset is structured to allow researchers to evaluate models on specific thresholds or across the entire cumulative set, facilitating the development of more robust, adaptive neuromorphic vision systems.
Experimental Results and Discussion
Analysis revealed a significant "noise floor" at low thresholds; at $T_4$, over 90% of pixel activations were background noise. Despite this, deep learning models showed remarkable resilience.
The "Goldilocks" Zone
Experimental results identified $T_{16}$ as the optimal threshold for most deep learning models, balancing detail retention with noise suppression. YOLOv4 and YOLOv7 emerged as the most robust performers. YOLOv4 achieved a peak AP50 of 97.20% at $T_8$, maintaining high accuracy even at the noisiest threshold ($T_4$).
Model Performance Comparison
- YOLOv4 and YOLOv7: Top performers, maintaining consistent accuracy across most thresholds.
- EfficientDet-b0: Exhibited high precision but suffered from low recall at higher thresholds ($T_{50}-T_{75}$), as the data became too sparse for reliable detection.
- MobileNet-v1: Performance was highly sensitive to the threshold, peaking sharply at $T_{16}$.
- HOG: Traditional feature extraction failed completely at low thresholds (0% AP50) due to an inability to distinguish object edges from pervasive noise.
Conclusion
The EBCD provides a vital tool for the neuromorphic vision community, highlighting that "more data" (lower thresholds) is not always beneficial due to noise. Future research will focus on expanding the dataset to include diverse weather conditions and developing adaptive neural networks capable of dynamically adjusting sensor thresholds in real-time to optimize power and performance in edge deployments.
Summary updated