Abstract
This paper presents a novel cascaded two-level algorithm for segmenting independently moving objects (IMOs) in event-based vision. Unlike traditional methods that often require a predefined number of objects or struggle with high computational costs, the proposed approach leverages a two-stage fitting scheme. First, sparse tracked features are clustered to generate robust motion hypotheses. Second, these hypotheses initialize a dense spatio-temporal graph cut to segment the entire event stream. The method achieves state-of-the-art accuracy on multiple benchmarks while operating 20 times faster than existing graph-cut techniques, demonstrating significant potential for real-time robotic applications.
Introduction
For autonomous systems to navigate dynamic environments, they must distinguish independently moving objects from the static background. While standard cameras face limitations like motion blur and restricted dynamic range, event cameras offer high temporal resolution and high dynamic range by asynchronously reporting per-pixel brightness changes. However, the sparse and asynchronous nature of event data poses significant algorithmic challenges. Existing motion segmentation methods often suffer from high computational complexity, sensitivity to noise, or the requirement of knowing the number of moving objects beforehand. This work addresses these issues through a cascaded multi-model fitting framework.
Methodology
The core of the proposed method is a cascaded two-level architecture designed to balance efficiency with pixel-level precision.
Level 1: Sparse Feature Clustering
The process begins by extracting and tracking sparse visual features (Shi-Tomasi corners) using optical flow. Instead of processing the entire event cloud, the algorithm clusters these sparse trajectories using a progressive multi-model fitting scheme (combining GC-RANSAC and PEARL optimization). This stage rapidly identifies a compact set of rigid motion models for both the background and any IMOs, effectively bootstrapping the segmentation process.
Level 2: Dense Event Clustering
The motion models from Level 1 serve as initializations for a dense spatio-temporal graph-cut algorithm. This stage assigns every event to a specific motion model by optimizing an energy function. This function incorporates data-model consistency via contrast maximization, spatial coherence to ensure neighboring events share labels, and a Minimum Description Length (MDL) penalty to prune redundant models. This cascaded approach allows the system to handle small objects that might otherwise be lost in background noise.
Experimental Evaluation
The algorithm was evaluated on several public datasets, including EED, EV-IMO, and DistSurf, alongside custom sequences featuring complex non-rigid motions.
Quantitative Results
The method demonstrated superior performance across all benchmarks. On the Extreme Event Dataset (EED), it achieved a 96.84% average detection rate, surpassing the previous state-of-the-art of 94.2%. On the EV-IMO dataset, it reached an 80.73% Intersection over Union (IoU). Crucially, the cascaded architecture enables a 20-fold speedup over previous graph-cut methods (EMSGC), with an average processing time of 0.36 seconds per segmentation.
Qualitative Robustness
The system exhibits remarkable robustness in challenging real-world scenarios, such as high-speed motion (e.g., a 1800 RPM fan) and high dynamic range conditions.
Conclusion
The cascaded two-level multi-model fitting approach provides a robust and efficient solution for event-based motion segmentation. By combining sparse feature tracking with dense graph-cut optimization, the method achieves high accuracy without requiring prior knowledge of the scene's complexity. While current limitations include the inability to distinguish objects with identical geometric motions, future work integrating semantic cues and non-linear motion models promises to further enhance performance in complex dynamic environments.
Summary updated