Introduction
Traditional frame-based computer vision often struggles in high-speed scenarios due to motion blur, or in high dynamic range environments where lighting is extreme. Event-based sensors, such as the Dynamic Vision Sensor (DVS), address these issues by reporting asynchronous pixel-level intensity changes with microsecond resolution. However, a significant challenge remains: detecting independent motion from a moving platform. When the camera moves, every pixel generates events, making it difficult to distinguish between background ego-motion and moving objects.
Methodology
The proposed framework treats the event stream as a 3D point cloud and applies a motion compensation pipeline to separate global camera motion from independent object motion.
Motion Model and Time-Image
The authors approximate camera motion using a 4-parameter model (x-shift, y-shift, radial expansion, and 2D rotation). A key innovation is the Time-Image ($\mathcal{T}$), a 2D grid where each pixel stores the average timestamp of events warped to that location. Unlike raw event counts, the Time-Image provides smooth gradients suitable for optimization.
Two-Stage Optimization
The pipeline employs a two-stage process:
- Coarse Alignment (Algorithm 1): Minimizes spatial gradients of the Time-Image to reduce temporal "blur."
- Fine Refinement (Algorithm 2): Maximizes event density in a standard event-count image to achieve sharpness.
Once the background is stabilized, any remaining "blurry" event clusters are identified as independently moving objects and tracked using a Kalman Filter.
Experimental Results
The framework was evaluated on the Extreme Event Dataset (EED), which includes challenging scenarios like fast-moving drones and strobe-lit environments.
Performance and Robustness
The system achieved detection success rates between 84.5% and 92.8%. Notably, it remained robust under extreme conditions, such as 1-2 Hz flashing strobe lights, where traditional cameras would fail. In terms of speed, a GPU implementation (NVIDIA Titan X) enabled processing at tens of thousands of frames per second equivalent, a 1000x-2300x speedup over CPU-based optimization.
Conclusion
This research demonstrates a robust method for real-time object tracking using event cameras by framing ego-motion compensation as a 3D alignment problem. While the system struggles with zero relative motion or extreme lack of background texture, it provides a high-performance solution for high-speed robotic navigation and surveillance.
Summary updated