Feedback Control of Event Cameras

Tobi Delbruck, Rui Graca, Marcin PaluchWorkshops @ CVPR 2021 · DOI

Abstract

Dynamic Vision Sensor (DVS) event cameras offer high-speed tracking and wide dynamic range by reporting pixel-level brightness changes. However, they suffer from data saturation in high-contrast scenes and noise in low-light conditions. This paper presents the first automated feedback control system for DVS cameras. By dynamically adjusting internal hardware biases—threshold, bandwidth, and refractory period—the system maintains event rates within an optimal range, ensuring reliable operation in unpredictable environments.

Introduction: The "Firehose" Problem

Event cameras represent a paradigm shift in computer vision, moving away from traditional frames to a sparse stream of asynchronous events. While this enables extreme performance, it introduces a significant challenge: the "firehose" effect. In high-motion or high-contrast scenarios, the event rate can surge to millions per second, overwhelming data buses and downstream processors. Conversely, static or low-light scenes may produce insufficient data or be dominated by sensor noise.

Historically, these cameras required manual tuning of internal "biases" (currents controlling pixel behavior), a process that is impractical for mobile robotics or autonomous systems operating in dynamic lighting. This work introduces a feedback control architecture to automate this tuning, keeping the event rate in a "Goldilocks" zone.

Overview of the automated feedback control system for DVS cameras, illustrating the loop between event stream statistics and hardware bias adjustments.

Methodology: Fixed-Step Feedback Control

The proposed system utilizes a fixed-step feedback controller. The design leverages the monotonic relationship between hardware biases and the resulting event rate—for instance, increasing the event threshold consistently decreases the event rate. This predictability allows for stable control without the need for complex modeling or deep learning.

The controller manages three primary parameters:

  1. Threshold Control: Adjusts the sensitivity to brightness changes. A state machine increases the threshold when the event rate $R$ exceeds an upper bound $R_H$ and decreases it when $R$ falls below $R_L$.
  2. Refractory Period Control: Enforces a "dead time" after a pixel fires, effectively capping the maximum event rate per pixel. This is particularly useful for limiting data volume during high motion without increasing noise.
  3. Bandwidth Control: Regulates the speed of the pixel's front-end. By monitoring background activity (noise), the controller adjusts the bandwidth to maintain a target noise level $R_{NL}$.
State machine logic for the three primary controllers: (A) threshold bounding, (B) refractory period limiting, and (C) noise-regulating bandwidth control.

Experimental Setup and Results

The system was evaluated using a DAVIS346 event camera under controlled and real-world conditions. Experiments included tracking a rotating dot on a monitor to vary speed and contrast, as well as capturing real-world driving footage.

Key Findings

  • Rate Bounding: The threshold controller successfully maintained a target event rate (e.g., 100–300 kHz) despite significant fluctuations in the input stimulus speed.
  • Noise Management: The bandwidth controller effectively suppressed "shot noise" in stationary scenes by automatically reducing pixel speed when no significant motion was detected.
  • Hardware Validation: The authors confirmed a linear relationship between temporal contrast sensitivity and signal event rate, validating the theoretical basis for the control loops.

Discussion and Future Work

The simplicity of the fixed-step approach ensures stability and ease of implementation. However, this method can be slow to react to instantaneous lighting changes, such as a vehicle entering a tunnel. Furthermore, the current controller is "scene-agnostic," treating all events with equal priority.

Future research could integrate semantic scene understanding to prioritize events from critical objects (e.g., pedestrians). Additionally, fusing bias control with other sensors, such as Inertial Measurement Units (IMUs), could allow the system to preemptively adjust parameters during rapid camera movements.

Summary updated

BibTeX
@inproceedings{delbruck2021feedback,
  author = {Tobi Delbruck and Rui Graca and Marcin Paluch},
  title = {{Feedback Control of Event Cameras}},
  booktitle = {Workshops @ CVPR},
  year = {2021},
  doi = {10.1109/cvprw53098.2021.00146},
}
Figure