Distance Surface for Event-Based Optical Flow

Mohammed Almatrafi, R. Wes Baldwin, Kiyoharu Aizawa, Keigo Hirakawa

Abstract

We propose DistSurf-OF, a novel optical flow method for neuromorphic cameras. Neuromorphic cameras (or event detection cameras) are an emerging sensor modality that makes use of dynamic vision sensors (DVS) to report asynchronously the log-intensity changes (called "events") exceeding a predefined threshold at each pixel. In absence of the intensity value at each pixel location, we introduce a notion of "distance surface"---the distance transform computed from the detected events---as a proxy for object texture. The distance surface is then used as an input to the intensity-based optical flow methods to recover the two dimensional pixel motion. Real sensor experiments verify that the proposed DistSurf-OF accurately estimates the angle and speed of each events.

The Challenge of Neuromorphic Data

Traditional computer vision relies on frame-based cameras that capture full intensity images at fixed intervals. In contrast, Dynamic Vision Sensors (DVS) operate asynchronously, triggering "events" only when a pixel detects a significant change in log-intensity. While this provides microsecond temporal resolution and high dynamic range, it discards the absolute intensity values required by the brightness constancy assumption—the cornerstone of most optical flow algorithms.

Comparison of APS vs DVS signals and the resulting spatio-temporal event cloud.

Previous attempts to solve event-based optical flow often suffered from the "aperture problem," where motion can only be detected perpendicular to edges, or relied on computationally expensive deep learning models that require massive amounts of labeled data.

The Distance Surface Framework

The core innovation of this work is the Distance Surface, $D(X, t)$. Instead of searching for intensity, the authors construct a continuous surface by calculating the distance from every pixel to the nearest event within a temporal window. This surface serves as a geometric proxy for object texture.

Crucially, the authors provide a mathematical proof demonstrating that this distance surface satisfies the fundamental optical flow equation: $$\nabla D \cdot V + D_t = 0$$ where $V$ is the velocity vector. Unlike traditional intensity-based equations which are first-order approximations, this distance surface equation is exact. This allows the use of classical, high-performance solvers like Horn-Schunck directly on event data.

The distance surface $D(X, k)$ and its spatial derivatives used for motion estimation.

To ensure robustness, the method includes a denoising step that categorizes events into Background Activity (BA), Inceptive Events (IE), and Trailing Events (TE). By excluding BA noise before surface construction, the algorithm maintains high accuracy even in noisy environments.

Experimental Evaluation

The researchers introduced the DVSMOTION20 dataset to benchmark the algorithm. Captured with an IniVation DAViS346 camera mounted on a gimbal, the dataset uses 6-axis IMU data to provide precise ground truth pixel velocity for stationary scenes.

DistSurf-OF demonstrated superior performance compared to other DVS-only methods:

  • Accuracy: It achieved an Average Angular Error (AAE) between 5.4° and 6.8°, significantly outperforming methods like EV-FlowNet (10°-18° AAE).
  • Efficiency: The distance surface computation is highly efficient, taking approximately 35ms per update.
  • Competitive Performance: Remarkably, DistSurf-OF performs nearly as well as hybrid methods (like DAViS-OF) that have access to both events and standard intensity frames.

Conclusion

DistSurf-OF bridges the gap between traditional frame-based algorithms and modern neuromorphic sensors. By transforming sparse events into a mathematically rigorous distance surface, it enables accurate, real-time optical flow estimation without the need for intensity data or complex training. Future work will explore hybrid models that combine distance surfaces with Active Pixel Sensor (APS) frames for even greater robustness.

Summary updated

Abstract

We propose DistSurf-OF, a novel optical flow method for neuromorphic cameras. Neuromorphic cameras (or event detection cameras) are an emerging sensor modality that makes use of dynamic vision sensors (DVS) to report asynchronously the log-intensity changes (called "events") exceeding a predefined threshold at each pixel. In absence of the intensity value at each pixel location, we introduce a notion of "distance surface"---the distance transform computed from the detected events---as a proxy for object texture. The distance surface is then used as an input to the intensity-based optical flow methods to recover the two dimensional pixel motion. Real sensor experiments verify that the proposed DistSurf-OF accurately estimates the angle and speed of each events.
BibTeX
@article{almatrafi2020distance,
  author = {Mohammed Almatrafi and R. Wes Baldwin and Kiyoharu Aizawa and Keigo Hirakawa},
  title = {{Distance Surface for Event-Based Optical Flow}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2020},
  url = {https://openreview.net/forum?id=cEZuswXMFjeG},
}
Figure