Live Demonstration: Real-Time Vi-SLAM With High-Resolution Event Camera

Gongyu Yang, Qilin Ye, Wanjun He, Lifeng Zhou, Xinyu Chen, Lei Yu, Wen Yang, Shoushun Chen, Wei LiWorkshops @ CVPR 2019 · OpenReview

Abstract

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes asynchronously. Compared to conventional frame-based cameras, they offer high dynamic range, low latency, and high sensitivity, making them ideal for SLAM in scenarios involving high-speed camera motion and low-light scenes. In this work, we implement real-time visual-inertial SLAM (VI-SLAM) using the recently released CeleX-V event camera. With its high spatial resolution (1280×800) and low latency (<0.5µs), the proposed method provides frames with abundant textures and high temporal response, leading to more stable tracking and improved performance in the SLAM system.

Introduction

Simultaneous Localization and Mapping (SLAM) is a cornerstone of autonomous systems. However, traditional frame-based cameras suffer from motion blur during high-speed maneuvers and lose detail in high-dynamic-range (HDR) or low-light environments. Event cameras, or "silicon retinas," address these issues by reporting pixel-level brightness changes asynchronously, resulting in microsecond-level latency and massive dynamic range.

Despite these advantages, early event cameras were limited by low spatial resolution (e.g., 240×180), which hindered the extraction of stable features necessary for accurate tracking. This paper introduces a real-time VI-SLAM pipeline that leverages the high-resolution CeleX-V sensor to overcome this "texture poverty," enabling robust localization in challenging conditions.

Methodology

The proposed system integrates high-definition event-based vision with an Inertial Measurement Unit (IMU) through a multi-stage pipeline.

System pipeline for the proposed VI-SLAM framework.

Sensor Fusion and Motion Compensation

The system utilizes the CeleX-V sensor, providing 1280×800 resolution events synchronized with a 125Hz IMU at the hardware level. To process the asynchronous event stream, the system synthesizes event frames within a spatio-temporal window. To mitigate motion blur within these frames, IMU data is used to calculate incremental transformations, allowing for the "de-blurring" of event positions through motion compensation.

Feature Detection and Tracking

Features are detected using the FAST corner detector on the motion-compensated frames. To ensure a uniform distribution of features, Non-Maximum Suppression (NMS) is applied, maintaining a minimum distance of 50 pixels between features.

Tracking is performed using a modified Lucas-Kanade (LK) optical flow. Standard LK often fails on event data; therefore, the system uses a 2D homography matrix derived from IMU rotation data to warp previous frames, facilitating more accurate feature matching. These matches are further validated using Normalized Product Correlation (NCC).

Backend Optimization

The backend employs a sliding-window optimization framework based on VINS-MONO. This stage refines the trajectory and map by fusing the tracked visual features with pre-integrated IMU measurements.

Key Results

The technical advantages of the CeleX-V sensor are significant when compared to previous generations of event cameras:

Camera Resolution Latency Bandwidth
DAVIS 240×180 3µs 12MHz
ATIS 304×320 3.2µs 30MHz
CeleX-V 1280×800 < 0.5µs 150MHz

The system was evaluated using a custom "Office Sequence" recorded with the CeleX-V. The high spatial resolution allows the system to maintain a high count of trackable features, which directly translates to superior localization precision.

Live demonstration of the SLAM system in an office sequence, showing event frames, standard frames, and the estimated trajectory.

As shown in the demonstration, the system maintains stable trajectory tracking throughout the environment. The motion-compensated event frames provide clear, high-contrast edges that are robust to lighting variations and rapid movements that typically degrade standard frame-based vision.

Conclusion

This work demonstrates that high-resolution event cameras are a game-changer for VI-SLAM. By moving to a 1280×800 resolution, the system gains the "abundant textures" required for reliable feature-based tracking. Future work will focus on optimizing the computational overhead associated with high-bandwidth event data and exploring purely asynchronous optimization methods to further exploit the unique properties of event-based sensors.

Summary updated

Abstract

Event camera is bio-inspired vision sensors that output pixel-level brightness changes asynchronously. Compared to the conventional frame-based camera, it is with high dynamic range, low latency and high sensitivity, and thus can be exploited in SLAM to tackle the problem of occasions with high-speed camera moving and low-light scenes. In this demo, we implement the visual-inerial SLAM in real time with the recently released event camera, namely, CeleX-V. With the feature of high spatial resolution (1280x800) and low latency <0.5us, the proposed method can provide frames with abundant textures and high time response, which leads to more stable tracking ability and better performance in SLAM system.
BibTeX
@inproceedings{yang2019live,
  author = {Gongyu Yang and Qilin Ye and Wanjun He and Lifeng Zhou and Xinyu Chen and Lei Yu and Wen Yang and Shoushun Chen and Wei Li},
  title = {{Live Demonstration: Real-Time Vi-SLAM With High-Resolution Event Camera}},
  booktitle = {Workshops @ CVPR},
  year = {2019},
  url = {https://openreview.net/forum?id=rVH0BYTusy},
}
Figure