Abstract: As a key technology of intelligent robots, visual sensors can achieve robot positioning, navigation, and target tracking. However, there is a lack of high-performance robot vision sensor platforms in the current market. This article introduces a powerful and efficient visual sensing platform called Guidance, which includes a central processing unit and multiple (five or more) servo sensing units. The built-in basic functions include visual positioning, obstacle avoidance, and depth of field image generation. In addition, Guidance comes with a complete SDK document that allows users to flexibly use and develop related projects, such as autonomous navigation, target tracking, positioning, and map building.
Introduction
Developed by researchers from DJI and the University of Science and Technology of China (USTC), Guidance is a high-performance, integrated visual sensing platform designed specifically for intelligent robots and drones. By combining five stereo camera units with a powerful FPGA-based processing core, Guidance provides real-time positioning, obstacle avoidance, and depth sensing, effectively bridging the gap between bulky consumer sensors and limited DIY solutions.
The Challenge of Robot Navigation
Navigating complex environments is a fundamental challenge for robots. While sensors like the Microsoft Kinect revolutionized indoor robotics, they are often too heavy, power-hungry, and reliant on infrared light, which fails in direct sunlight. Existing lightweight alternatives at the time of Guidance's development, such as PX4FLOW or single-direction VI-Sensors, suffered from low resolution, narrow fields of view, or excessive computational burdens that reduced flight time and payload capacity for small drones.
System Architecture
The authors propose a hardware-software co-design to handle the massive data throughput of ten cameras simultaneously.
Hardware Architecture
- Sensing Units (SSU): The system utilizes five units, each containing two global-shutter grayscale cameras (640x480) and one ultrasonic sensor. These are positioned to provide a wide, multi-directional field of view.
- Processing Core: An Altera Cyclone V SoC FPGA allows for parallel hardware acceleration of image processing tasks, such as feature extraction and stereo matching, while an ARM core handles high-level logic.
Algorithmic Pipeline
The software architecture is designed for high-speed data fusion and motion estimation:
- Visual Odometry (VO): Employs FAST corner detection and BRIEF descriptors with a "non-pyramid" Lucas-Kanade tracker for matching.
- Motion Estimation & Scale: The system uses 2D-2D geometry constraints. To overcome monocular scale limitations, it integrates an IMU and ultrasonic data through an Extended Kalman Filter (EKF).
- Mapping: The system generates depth maps at 20Hz and utilizes a multi-scale Octomap approach to store 3D data efficiently.
Key Insights and Results
- Hardware Acceleration: Offloading image processing to an FPGA enables a processing speed of approximately 47.34ms, allowing for real-time control loops.
- Multi-modal Redundancy: Ultrasonic sensors act as a fail-safe for vision in environments with glass or low-texture surfaces.
- High Precision: In outdoor tests, the system provided velocity estimates with a standard deviation of only 0.08 m/s compared to GPS ground truth.
- Obstacle Avoidance: Guidance provides a reliable sensing range of 0.1 to 20 meters.
Conclusion
Guidance represents a significant step forward in providing a robust, real-time visual sensing platform for autonomous systems. While currently limited by grayscale imaging and specific environmental blind spots, its ability to fuse multi-modal sensor data with hardware-accelerated processing proves highly effective for complex autonomous missions in both indoor and outdoor settings.
Summary updated