Frugal event data: how small is too small? A human performance assessment with shrinking data

Amélie Gruel, Lucía Trillo Carreras, Marina Bueno García, Ewa Kupczyk, Jean MartinetWorkshops @ CVPR 2023 · DOI

Introduction

Event cameras, or Dynamic Vision Sensors (DVS), represent a paradigm shift in computer vision by recording only changes in brightness at microsecond temporal resolution. While this offers high dynamic range and low latency, the resulting high-frequency data stream can overwhelm the computational budgets of embedded systems. To achieve "frugal" event-based vision for low-power devices, data must be downscaled. This paper investigates the lower bound of information preservation by using human perception as the ultimate benchmark for gesture recognition in downscaled event data.

Methodology

Downscaling Techniques

The researchers evaluated two primary categories of data reduction:

  1. Spatial Downscaling: Reducing the original $128 \times 128$ pixel grid to resolutions as low as $13 \times 13$. Methods included fast spatial funnelling, event count estimation, log-luminance reconstruction (Linear and Cubic), and biologically inspired Spiking Neural Network (SNN) pooling.
  2. Temporal Downscaling: Reducing event density over time through temporal funnelling (grouping events into windows) and structural downscaling (deterministic or stochastic event dropping).

Human Perception Study

A study was conducted with 60 participants to classify four specific gestures from the IBM DVS128 dataset: right hand clockwise, right hand counter-clockwise, "drums," and "hand rolls." These gestures were selected for their spatial similarity, requiring high temporal clarity to distinguish correctly.

The four gestures used in the study: (A) Right hand clockwise, (B) Right hand counter-clockwise, (C) Drums, and (D) Hand rolls.

Participants were presented with 2D visualizations of the downscaled data and asked to identify the gesture or indicate if they could not recognize it.

The experimental interface used for the human perception task, displaying a downscaled event sample alongside multiple-choice options.

Key Insights and Results

The Failure of Visual Reconstruction

Surprisingly, methods that attempted to reconstruct "pretty," video-like intensity signals (log-luminance reconstruction) performed the worst in human trials. This suggests that the raw, sparse spikes of event data contain vital motion cues that are lost when smoothed into traditional grayscale representations.

SNNs and Information Preservation

SNN-based pooling methods emerged as top performers, maintaining high human accuracy even at significant compression levels. This highlights the efficiency of neuromorphic architectures in retaining semantic features while reducing data volume.

Critical Thresholds

  • Spatial Limit: Human accuracy typically drops below the chance level (25%) when the spatial dividing factor reaches 10 (approximately $13 \times 13$ resolution).
  • Temporal Limit: For temporal funnelling, recognition plummeted when time windows exceeded 0.5 seconds. However, in structural downscaling, humans could still identify gestures until data was reduced to a mere 0.1% of its original volume.
  • Stochastic Advantage: In temporal reduction, dropping events randomly (stochastic) preserved better recognition than deterministic dropping, as it better maintained the overall temporal structure of the motion.

Conclusion

By benchmarking against human performance, this research establishes that event-based data can be significantly compressed—down to 0.1% of its original volume in some cases—before losing its semantic utility. The findings suggest that future hardware-friendly neural networks for edge computing should prioritize neuromorphic pooling and stochastic sampling to maintain high performance in data-constrained environments.

Summary updated

BibTeX
@inproceedings{gruel2023frugal,
  author = {Amélie Gruel and Lucía Trillo Carreras and Marina Bueno García and Ewa Kupczyk and Jean Martinet},
  title = {{Frugal event data: how small is too small? A human performance assessment with shrinking data}},
  booktitle = {Workshops @ CVPR},
  year = {2023},
  doi = {10.1109/cvprw59228.2023.00430},
}
Figure