A Reconfigurable Visual-Inertial Odometry Accelerated Core with High Area and Energy Efficiency for Autonomous Mobile Robots

Yonghao Tan, Mengying Sun, Huanshihong Deng, Haihan Wu, Minghao Zhou, Yifei Chen, Zhuo Yu, Qinghan Zeng, Ping Li, Lei Chen, Fengwei An2022 · DOI

TL;DR

This paper presents a novel, reconfigurable hardware accelerator for Visual-Inertial Odometry (VIO) designed for autonomous mobile robots. It achieves significant improvements in area and energy efficiency while maintaining high accuracy, making it ideal for resource-constrained robotic platforms.

What Problem Does This Paper Address?

Autonomous mobile robots (AMRs) rely heavily on accurate and efficient localization and mapping (SLAM) systems. Visual-Inertial Odometry (VIO), which fuses camera and inertial measurement unit (IMU) data, is a popular approach for this. However, existing VIO systems face challenges: the algorithms are computationally complex, dedicated hardware accelerators are often inflexible and have long development cycles, and AMRs typically have limited power budgets. This leads to a trade-off between accuracy, speed, and energy consumption. The authors aim to bridge this gap by designing a reconfigurable VIO accelerator that is both highly efficient and adaptable to different VIO algorithms.

Methods

The core of this work is a reconfigurable hardware accelerator designed for FPGAs. Key methodological aspects include:

  • Reconfigurable Architecture: The accelerator is designed to support multiple VIO algorithms, offering flexibility that is often lacking in dedicated hardware. This is achieved through a programmable computation core with a custom instruction set architecture (ISA).
  • Optimized Workflow: The system integrates IMU pre-integration, feature extraction, and Extended Kalman Filter (EKF) based optimization. It employs a pipelined architecture with line buffers to reduce memory consumption and increase throughput.
  • Efficient Feature Extraction: The paper utilizes a modified FAST-9 feature detection algorithm, balancing speed and accuracy. It also incorporates a gradient-based score to assess feature quality and a patch extraction strategy to avoid feature clustering.
  • Memory Management: A key innovation is the use of a shared memory strategy and memory reuse, resulting in a remarkably low on-chip memory footprint (70 KB).
  • Hardware-Software Co-design: The ISA and hardware modules are designed to work in tandem, enabling efficient execution of VIO tasks.

Datasets

The paper evaluates the accuracy of the proposed VIO accelerator using the EuRoC dataset. This is a well-established and challenging dataset for VIO and SLAM research. The authors also mention translating the ROVIO algorithm into their proposed ISA for comparison.

Key Insights and Ideas

  • Reconfigurability is Key: The ability to adapt the hardware to different VIO algorithms is a significant advantage, allowing for future-proofing and broader applicability without redesigning the entire chip.
  • Memory Footprint Reduction: Achieving a 70 KB on-chip memory usage is a major breakthrough, drastically reducing the hardware complexity and power requirements compared to prior works. This is achieved through clever memory reuse and an efficient pipeline.
  • EKF Optimization for Efficiency: The choice of an EKF-based backend, rather than more complex methods like bundle adjustment, is crucial for reducing computational load and memory requirements, aligning with the goal of lightweight AMRs.
  • Hardware-Friendly FAST: Adapting the FAST algorithm for hardware implementation with pipeline stages and specific logic for brightness comparison demonstrates a practical approach to accelerating a common VIO component.

Key Results

  • Accuracy: The proposed VIO accelerator shows negligible accuracy loss compared to the software-based ROVIO algorithm on the EuRoC dataset.
  • On-chip Memory: The accelerator uses only 70 KB of on-chip memory, which is at least 10x smaller than state-of-the-art works.
  • Hardware Resources: FPGA implementation shows significantly lower LUTs and FFs consumption compared to previous FPGA implementations.
  • Area Efficiency: In 28 nm CMOS technology, the core area is a mere 3 mm², which is substantially smaller than comparable ASIC implementations.
  • Power Efficiency: The power consumption is competitive, with the FPGA platform consuming about 0.683 W and the 28 nm CMOS synthesis achieving 179.52 mW.

Limitations and Future Work

  • Dataset Specificity: While the EuRoC dataset is authoritative, the performance on other, potentially more challenging or diverse, datasets is not explicitly explored.
  • Algorithm Support: The paper focuses on EKF-based VIO algorithms. While reconfigurable, extending support to other SLAM paradigms (e.g., feature-less methods, different optimization backends) could be a future direction.
  • Real-world Deployment: While an evaluation platform is shown, extensive real-world testing in highly dynamic or challenging environments (e.g., extreme lighting, rapid motion) would further validate its robustness.

The authors highlight the potential for further downstream exploration due to the reconfigurability of the core, suggesting that this architecture could be a foundation for a wider range of AMR applications.

Summary updated

BibTeX
@misc{tan2022reconfigurable,
  author = {Yonghao Tan and Mengying Sun and Huanshihong Deng and Haihan Wu and Minghao Zhou and Yifei Chen and Zhuo Yu and Qinghan Zeng and Ping Li and Lei Chen and Fengwei An},
  title = {{A Reconfigurable Visual-Inertial Odometry Accelerated Core with High Area and Energy Efficiency for Autonomous Mobile Robots}},
  year = {2022},
  doi = {10.3390/s22197669},
}
Figure