TL;DR
This paper presents a novel, low-cost ($<2000 USD, ~1kg) mobile mapping system built around an Android smartphone and a Livox lidar. It achieves good 3D reconstruction and mapping accuracy, making advanced reality capture accessible and environmentally friendly by repurposing existing smartphone technology.
What Problem Does This Paper Address?
The metaverse, digital twins, and robotics all rely on accurate 3D models of the real world. While professional mobile mapping systems exist (e.g., Leica BLK2GO), they are often prohibitively expensive or bulky. Existing low-cost solutions might compromise on range or accuracy. This paper tackles the challenge of creating an affordable, portable, and accurate mobile mapping system by leveraging the ubiquitous computing power and advanced sensors already present in modern Android smartphones, combined with a cost-effective lidar. This approach aims to democratize access to 3D mapping technology.
Methods
The core of the system is an Android application that integrates data from a Livox Mid360 lidar, the smartphone's camera and IMU, and an external GNSS-RTK stick for georeferencing. Key technical contributions include:
- Lidar-Inertial Odometry (LIO): The system utilizes the Fast-LIO2 algorithm, implemented in C++ via the Native Development Kit (NDK) and ROSJava for Android integration, to provide real-time pose tracking.
- Multisensor Calibration: A comprehensive calibration procedure is detailed, covering wide-angle camera intrinsics, spatio-temporal calibration between the rolling shutter camera and IMU, and temporal calibration between the lidar and IMU.
- Data Logging: A module efficiently logs synchronized data from all sensors (lidar, camera, IMU, GNSS-RTK) with precise timestamping.
- Open-Source Design: The system's hardware design and software are made publicly available, fostering community development and adoption.
Datasets
The paper evaluates the system using datasets collected from two pre-owned smartphones (Redmi K60 Pro and Samsung Galaxy S22+) in various environments, including a basement parking lot and outdoor urban areas. Ground truth data was generated using a Leica RTC360 terrestrial laser scanner (TLS) and processed with CloudCompare for comparison. No new benchmark datasets were introduced; instead, existing real-world scenarios were used for evaluation.
Key Insights and Ideas
- Smartphone as a Powerful Platform: The paper convincingly demonstrates that modern smartphones, with their advanced sensors and processing capabilities, can serve as the central hub for sophisticated mobile mapping systems, significantly reducing costs.
- Repurposing E-Waste: By utilizing discarded or older smartphones, the system offers an environmentally friendly approach to reality capture, addressing the growing issue of electronic waste.
- Importance of Precise Calibration: The results highlight that accurate spatio-temporal calibration between the rolling shutter camera, IMU, and lidar is crucial for achieving high-quality odometry and mapping. Even small time offsets can impact performance.
- Real-time Performance on Mobile: The successful implementation of Fast-LIO2 on a smartphone, achieving near real-time performance, is a significant achievement, showcasing the potential of edge computing for robotics.
Key Results
- Affordability and Portability: The system boasts a Bill of Materials (BOM) cost under $2,000 USD and a weight of approximately 1 kilogram, making it highly accessible and easy to handle.
- Tracking Accuracy: The online Faster-LIO implementation achieved remarkable accuracy, closely matching offline lidar-inertial odometry methods like Fast-LIO2. Relative pose errors were consistently low (e.g., <0.4% translation error).
- Mapping Accuracy: While cloud-to-cloud distances between the system's generated maps and TLS ground truth were sometimes up to 20 cm, the system produced visually consistent point clouds, with wall thickness generally within 5 cm.
- Runtime Efficiency: The Faster-LIO algorithm ran in near real-time on the tested smartphones, with frame processing times well within acceptable limits for interactive use.
Limitations and Future Work
- Rolling Shutter Effect: The paper acknowledges that the rolling shutter effect of smartphone cameras negatively impacts the performance of visual-inertial odometry methods like Fast-LIVO2, which assume a global shutter. This is a key limitation for vision-heavy approaches.
- Lidar Range Limitations: The Livox Mid360 lidar's accuracy decreases with range, making it less suitable for mapping very large areas or distant objects.
- IP Address Management: A minor limitation is the need to reconfigure the lidar's IP address after each reboot, requiring manual intervention or a more robust connection management solution.
- Battery Consumption: Connecting the GNSS-RTK stick via Bluetooth increases the smartphone's battery drain.
Future work could involve integrating longer-range lidar sensors, exploring more advanced loop closure techniques (e.g., BALM2), and developing more robust methods to handle the rolling shutter effect. Further optimization of the software for even greater efficiency on resource-constrained devices is also a promising direction.
Summary updated