Guidance - Robot Vision Sensing Platform

Guyue Zhou, Ketan Tang, Honghui Zhang, Kai Wang, Kang Yang, Lu FangPrestress 2016 · DOI

Abstract: As a key technology of intelligent robots, visual sensors can achieve robot positioning, navigation, and target tracking. However, there is a lack of high-performance robot vision sensor platforms in the current market. This article introduces a powerful and efficient visual sensing platform called Guidance, which includes a central processing unit and multiple (five or more) servo sensing units. The built-in basic functions include visual positioning, obstacle avoidance, and depth of field image generation. In addition, Guidance comes with a complete SDK document that allows users to flexibly use and develop related projects, such as autonomous navigation, target tracking, positioning, and map building.

Introduction

Developed by researchers from DJI and the University of Science and Technology of China (USTC), Guidance is a high-performance, integrated visual sensing platform designed specifically for intelligent robots and drones. By combining five stereo camera units with a powerful FPGA-based processing core, Guidance provides real-time positioning, obstacle avoidance, and depth sensing, effectively bridging the gap between bulky consumer sensors and limited DIY solutions.

The Challenge of Robot Navigation

Navigating complex environments is a fundamental challenge for robots. While sensors like the Microsoft Kinect revolutionized indoor robotics, they are often too heavy, power-hungry, and reliant on infrared light, which fails in direct sunlight. Existing lightweight alternatives at the time of Guidance's development, such as PX4FLOW or single-direction VI-Sensors, suffered from low resolution, narrow fields of view, or excessive computational burdens that reduced flight time and payload capacity for small drones.

System Architecture

The authors propose a hardware-software co-design to handle the massive data throughput of ten cameras simultaneously.

Hardware Architecture

  • Sensing Units (SSU): The system utilizes five units, each containing two global-shutter grayscale cameras (640x480) and one ultrasonic sensor. These are positioned to provide a wide, multi-directional field of view.
  • Processing Core: An Altera Cyclone V SoC FPGA allows for parallel hardware acceleration of image processing tasks, such as feature extraction and stereo matching, while an ARM core handles high-level logic.
The Guidance system integrated onto a drone platform for real-world testing.

Algorithmic Pipeline

The software architecture is designed for high-speed data fusion and motion estimation:

  • Visual Odometry (VO): Employs FAST corner detection and BRIEF descriptors with a "non-pyramid" Lucas-Kanade tracker for matching.
  • Motion Estimation & Scale: The system uses 2D-2D geometry constraints. To overcome monocular scale limitations, it integrates an IMU and ultrasonic data through an Extended Kalman Filter (EKF).
  • Mapping: The system generates depth maps at 20Hz and utilizes a multi-scale Octomap approach to store 3D data efficiently.
The multi-layered algorithmic pipeline showing data flow from sensing units to the local system.

Key Insights and Results

  • Hardware Acceleration: Offloading image processing to an FPGA enables a processing speed of approximately 47.34ms, allowing for real-time control loops.
  • Multi-modal Redundancy: Ultrasonic sensors act as a fail-safe for vision in environments with glass or low-texture surfaces.
  • High Precision: In outdoor tests, the system provided velocity estimates with a standard deviation of only 0.08 m/s compared to GPS ground truth.
  • Obstacle Avoidance: Guidance provides a reliable sensing range of 0.1 to 20 meters.

Conclusion

Guidance represents a significant step forward in providing a robust, real-time visual sensing platform for autonomous systems. While currently limited by grayscale imaging and specific environmental blind spots, its ability to fuse multi-modal sensor data with hardware-accelerated processing proves highly effective for complex autonomous missions in both indoor and outdoor settings.

Summary updated

Abstract

s a key technology of intelligent robots, visual sensors can achieve robot positioning, navigation, and target tracking. However, there is a lack of high-performance robot vision sensor platforms in the current market. This article will introduce a powerful and efficient visual sensing platform called Guidance, which includes a central processing unit and multiple (five or more) servo sensing units. The built-in basic functions include visual positioning, obstacle avoidance, and depth of field image generation. In addition, Guidance comes with a complete SDK document that allows users to flexibly use and develop related projects, such as autonomous navigation, target tracking, positioning, and map building
BibTeX
@inproceedings{zhou2016guidance,
  author = {Guyue Zhou and Ketan Tang and Honghui Zhang and Kai Wang and Kang Yang and Lu Fang},
  title = {{Guidance - Robot Vision Sensing Platform}},
  booktitle = {Prestress},
  year = {2016},
  doi = {10.59238/j.pt.2016.03.003},
}
Figure