Live Demonstration: CeleX-V: A 1M Pixel Multi-Mode Event-Based Sensor

Shoushun Chen, Menghan GuoWorkshops @ CVPR 2019 · DOI

Abstract

The CeleX-V represents a significant advancement in neuromorphic engineering, featuring a 1-megapixel event-based sensor that integrates high-speed motion detection with traditional frame-based imaging. By providing on-chip optical flow extraction and a hybrid pixel architecture, the CeleX-V offers a low-power, high-resolution solution for real-time machine vision applications, bridging the gap between dynamic vision sensors and standard CMOS cameras.

Introduction

Traditional frame-based cameras often suffer from data redundancy and high latency when capturing fast-moving scenes. Event-based sensors, or Dynamic Vision Sensors (DVS), mitigate these issues by only reporting changes in pixel intensity. However, widespread adoption has been hindered by low resolutions, a lack of static intensity context, and high external processing requirements for motion analysis.

The CeleX-V addresses these challenges by delivering a 1280 × 800 pixel resolution sensor capable of multi-mode output. It provides both event-driven data and full-frame intensity information while performing complex tasks like optical flow extraction directly on the chip.

Methods and Architecture

Fabricated using a 65nm CMOS Image Sensor (CIS) process, the CeleX-V utilizes a hybrid pixel architecture. Each pixel incorporates a logarithmic photodetector to capture intensity without additional exposure time. The sensor outputs data in a flexible (x, y, a, t) format, where a represents the pixel's intensity, allowing for simultaneous motion sensing and standard imaging.

Figure 1: Chip layout of the CeleX-V sensor showing the 1280x800 pixel array and on-chip processing units.

Key architectural features include:

  • On-Chip Processing: Full-array-parallel motion detection and on-chip optical flow extraction are performed locally, reducing the computational burden on external processors.
  • High-Speed Interfaces: A serial MIPI interface (up to 2.4Gbps) and a parallel interface (140M pixels/sec) manage the high data throughput of the 1MP array.
  • Environmental Robustness: Integrated digital temperature and ambient light sensors ensure consistent performance across varying operating conditions.

Demonstration and Results

The sensor's capabilities were demonstrated through real-world scenarios, such as tracking a moving vehicle. The CeleX-V successfully produced high-resolution grayscale frames alongside event-based reconstructions and real-time optical flow visualizations.

Figure 2: Comprehensive demonstration of CeleX-V outputs, including chip layout, standard grayscale frame, event-based reconstruction, and color-coded optical flow.

The demonstration highlighted several key performance metrics:

  • Resolution: 1-megapixel (1280 × 800) with a 9.8µm pixel pitch.
  • Power Efficiency: Typical consumption of 400mW despite intensive on-chip processing.
  • Temporal Resolution: Reconstructed video from events at 30ms intervals and optical flow at 60ms intervals, showing clear motion gradients and object tracking.

Discussion and Insights

The CeleX-V's "best of both worlds" approach—integrating intensity data into event packets—ensures compatibility with legacy computer vision algorithms while maintaining the benefits of event-based sensing. By moving optical flow extraction to the "edge" (the sensor itself), the system significantly reduces bandwidth and power requirements for host systems. This milestone in resolution and integrated intelligence marks a significant step forward for autonomous systems and high-speed industrial inspection.

Conclusion

The CeleX-V demonstrates that high-resolution, intelligent event-based sensing is viable for modern machine vision. By combining the speed of neuromorphic hardware with the detail of traditional imaging, it provides a versatile platform for the next generation of edge AI applications.

Summary updated

BibTeX
@inproceedings{chen2019live,
  author = {Shoushun Chen and Menghan Guo},
  title = {{Live Demonstration: CeleX-V: A 1M Pixel Multi-Mode Event-Based Sensor}},
  booktitle = {Workshops @ CVPR},
  year = {2019},
  doi = {10.1109/cvprw.2019.00214},
}
Figure