Abstract
Event-based cameras offer ultra-low latency and high dynamic range, yet extracting full 2D optical flow from their asynchronous, sparse data remains a significant challenge. This paper introduces Tangentially Elongated Gaussian Belief Propagation (TEGBP), a novel framework that bridges the gap between fast, local normal-flow estimators and accurate, global dense models. By representing motion uncertainty through Tangentially Elongated Gaussians (TEGs) and performing incremental inference on a factor graph, TEGBP achieves state-of-the-art accuracy while maintaining the computational efficiency required for real-time, event-driven applications.
Introduction
Traditional frame-based cameras are inherently limited by fixed exposure times and frame rates, which often lead to motion blur or data redundancy. In contrast, event-based sensors, such as Dynamic Vision Sensors (DVS), detect intensity changes asynchronously at each pixel, providing a stream of events with microsecond-level temporal resolution. While these sensors are ideal for high-speed motion analysis, they suffer from the fundamental "aperture problem": a single event only provides information about motion perpendicular to a local edge, known as normal flow.
Recovering the "full flow"—the true 2D velocity of a pixel—typically requires integrating information across multiple edges or orientations. Existing solutions often fall into two categories: fast incremental algorithms that struggle with accuracy due to the aperture problem, and high-accuracy deep learning models that are non-incremental and computationally expensive. The authors propose TEGBP to address this trade-off, providing a sparse, incremental, and accurate solution for full flow estimation.
The TEGBP Framework
The core innovation of this work is the Tangentially Elongated Gaussian (TEG) model. Instead of treating a normal flow measurement as a deterministic vector, the authors model it as a 2D Gaussian distribution in velocity space.
As shown in Figure 1, the TEG is designed with a very low variance in the direction perpendicular to the edge (where the measurement is certain) and a high variance along the edge (where the aperture problem introduces uncertainty). This probabilistic representation allows the system to maintain the full 2D uncertainty of each event.
To estimate the global flow, the image is represented as a sparse factor graph. Pixels act as variable nodes, and events introduce measurement factors. A smoothness prior is enforced via prior factors between neighboring nodes, assuming that local motion is spatially consistent. The system uses Belief Propagation (BP) to pass messages between nodes, incrementally updating the "belief" of the flow at each pixel as new events arrive.
Key Insights and Probabilistic Modeling
The effectiveness of TEGBP stems from its ability to handle the aperture problem through marginalization. When multiple TEGs from different edge orientations are combined with a smoothness prior, the joint distribution naturally converges toward the true full flow. This approach avoids the pitfalls of naive averaging used in previous incremental methods, which often leads to blurred or incorrect velocity estimates.
Furthermore, the algorithm leverages the inherent sparsity of event data. By only performing computations at the locations of active events, TEGBP preserves the low-latency benefits of the event sensor. A coarse-to-fine scheme is also employed to handle large-scale motions and accelerate the propagation of information across the factor graph.
Experimental Evaluation
The authors evaluated TEGBP against several benchmarks, including simulated datasets (ESIM-bricks) and real-world sequences (MVSEC, DSEC, and DVS-stripes). The results demonstrate a significant performance leap over the previous state-of-the-art incremental method, ARMS.
On the MVSEC dataset, TEGBP reduced the Average Endpoint Error (AEE) by approximately 30-40% compared to ARMS. For instance, in the indoor_flying1 sequence, the AEE dropped from 1.71 to 1.14. Additionally, the percentage of outlier pixels was halved in many scenarios. Qualitative tests on the DVS-stripes sequence showed that TEGBP could successfully distinguish the velocities of adjacent objects with different motions, a task where previous methods failed due to over-smoothing.
Conclusion and Future Work
TEGBP represents a major step forward in event-based optical flow, offering a robust and efficient way to recover full 2D motion. However, some limitations remain. The method's accuracy is partially dependent on the quality of the initial normal flow estimate; noisy textures can degrade performance. Future research will explore the use of lightweight neural networks to refine these initial measurements and the implementation of discontinuity-aware smoothness priors to better handle object boundaries. Finally, demonstrating the algorithm on specialized data-flow hardware will be crucial for proving its real-world utility in edge computing environments.
Summary updated