Abstract
Event cameras are bio-inspired vision sensors that output pixel-level brightness changes asynchronously. Compared to conventional frame-based cameras, they offer high dynamic range, low latency, and high sensitivity, making them ideal for SLAM in scenarios involving high-speed camera motion and low-light scenes. In this work, we implement real-time visual-inertial SLAM (VI-SLAM) using the recently released CeleX-V event camera. With its high spatial resolution (1280×800) and low latency (<0.5µs), the proposed method provides frames with abundant textures and high temporal response, leading to more stable tracking and improved performance in the SLAM system.
Introduction
Simultaneous Localization and Mapping (SLAM) is a cornerstone of autonomous systems. However, traditional frame-based cameras suffer from motion blur during high-speed maneuvers and lose detail in high-dynamic-range (HDR) or low-light environments. Event cameras, or "silicon retinas," address these issues by reporting pixel-level brightness changes asynchronously, resulting in microsecond-level latency and massive dynamic range.
Despite these advantages, early event cameras were limited by low spatial resolution (e.g., 240×180), which hindered the extraction of stable features necessary for accurate tracking. This paper introduces a real-time VI-SLAM pipeline that leverages the high-resolution CeleX-V sensor to overcome this "texture poverty," enabling robust localization in challenging conditions.
Methodology
The proposed system integrates high-definition event-based vision with an Inertial Measurement Unit (IMU) through a multi-stage pipeline.
Sensor Fusion and Motion Compensation
The system utilizes the CeleX-V sensor, providing 1280×800 resolution events synchronized with a 125Hz IMU at the hardware level. To process the asynchronous event stream, the system synthesizes event frames within a spatio-temporal window. To mitigate motion blur within these frames, IMU data is used to calculate incremental transformations, allowing for the "de-blurring" of event positions through motion compensation.
Feature Detection and Tracking
Features are detected using the FAST corner detector on the motion-compensated frames. To ensure a uniform distribution of features, Non-Maximum Suppression (NMS) is applied, maintaining a minimum distance of 50 pixels between features.
Tracking is performed using a modified Lucas-Kanade (LK) optical flow. Standard LK often fails on event data; therefore, the system uses a 2D homography matrix derived from IMU rotation data to warp previous frames, facilitating more accurate feature matching. These matches are further validated using Normalized Product Correlation (NCC).
Backend Optimization
The backend employs a sliding-window optimization framework based on VINS-MONO. This stage refines the trajectory and map by fusing the tracked visual features with pre-integrated IMU measurements.
Key Results
The technical advantages of the CeleX-V sensor are significant when compared to previous generations of event cameras:
| Camera | Resolution | Latency | Bandwidth |
|---|---|---|---|
| DAVIS | 240×180 | 3µs | 12MHz |
| ATIS | 304×320 | 3.2µs | 30MHz |
| CeleX-V | 1280×800 | < 0.5µs | 150MHz |
The system was evaluated using a custom "Office Sequence" recorded with the CeleX-V. The high spatial resolution allows the system to maintain a high count of trackable features, which directly translates to superior localization precision.
As shown in the demonstration, the system maintains stable trajectory tracking throughout the environment. The motion-compensated event frames provide clear, high-contrast edges that are robust to lighting variations and rapid movements that typically degrade standard frame-based vision.
Conclusion
This work demonstrates that high-resolution event cameras are a game-changer for VI-SLAM. By moving to a 1280×800 resolution, the system gains the "abundant textures" required for reliable feature-based tracking. Future work will focus on optimizing the computational overhead associated with high-bandwidth event data and exploring purely asynchronous optimization methods to further exploit the unique properties of event-based sensors.
Summary updated