LiDARTag: A Real-Time Fiducial Tag System for Point Clouds

Jiunn-Kai Huang, Shoutian Wang, Maani Ghaffari, Jessy W. GrizzleIEEE RA-L 2021 · DOI

TL;DR

This paper introduces LiDARTag, a novel fiducial marker system designed for LiDAR point clouds, enabling robust real-time detection and pose estimation even in complete darkness. By adapting existing visual tag designs and employing a unique point cloud processing pipeline, LiDARTag offers a complementary solution to camera-based markers for multi-sensor fusion and calibration.

What Problem Does This Paper Address?

Traditional fiducial markers, like AprilTags, are widely used for tasks such as object tracking, camera calibration, and SLAM. However, these markers rely on visual cues and are highly sensitive to ambient lighting conditions, failing in low-light or dark environments. LiDAR sensors, on the other hand, are active sensors and are inherently robust to lighting changes. The challenge is that existing fiducial markers are not designed to be detected by LiDARs, and LiDAR point clouds are inherently unstructured and sparse, making it difficult to isolate and identify markers. This paper addresses the need for a fiducial marker system that can be reliably detected by LiDARs, complementing visual markers and enabling robust performance in challenging lighting conditions.

Methods

The LiDARTag system comprises several key components:

  • Tag Design: LiDARTags are based on existing visual fiducial markers (e.g., AprilTags) but are designed to have varying reflectivity. This allows them to be detected by both cameras and LiDARs. The payload of the tag is composed of differing reflectivities, placed within a defined boundary.
  • Feature Detection: The system identifies "edge points" in the LiDAR point cloud by analyzing discontinuities in distance. This is achieved by computing spatial gradients of the distance along each beam.
  • Feature Clustering: Detected features are grouped into clusters using a single-linkage agglomerative hierarchical clustering algorithm. This approach is chosen because the number of LiDARTags is unknown, making methods like K-Means unsuitable.
  • Cluster Validation: Clusters are validated using heuristics based on the number of points and features, as well as a plane fitting process to identify outlier percentages. This step filters out clusters that are unlikely to contain a LiDARTag.
  • Pose Estimation: The pose of a LiDARTag is estimated by minimizing an L1-inspired fitting error between the point cloud returns from the tag and a known marker template. This optimization problem is solved using a gradient-based solver.
  • ID Decoding: To overcome the sparsity of LiDAR point clouds, the projected point cloud is lifted into a continuous function within a Reproducing Kernel Hilbert Space (RKHS). The inner product between this function and pre-computed functions for each possible tag ID is used to determine the marker's ID. This RKHS approach also helps to remove rotational ambiguities.

Datasets

The paper evaluates the LiDARTag system using the following datasets:

  • Google Cartographer indoor dataset: Used for validating false positive rejection in cluttered indoor environments.
  • Honda H3D outdoor dataset: Used for validating false positive rejection in spacious outdoor environments.
  • Custom datasets: Collected using a 32-Beam Velodyne ULTRA Puck LiDAR and an Intel RealSense camera attached to a robot, with ground truth provided by a motion capture system. These datasets include scenarios with tags at various distances and rotations.

Key Insights and Ideas

  • Leveraging Reflectivity for LiDAR Detection: The core idea of using varying reflectivity on a planar marker is a simple yet effective way to make fiducial markers detectable by active LiDAR sensors.
  • RKHS for Sparse Point Cloud Data: Lifting sparse LiDAR point clouds into an RKHS is a powerful technique to create continuous representations, enabling robust feature extraction and ID decoding even with limited data points.
  • Complementary to Visual Markers: LiDARTag is designed to be compatible with existing visual fiducial markers, opening doors for efficient multi-sensor fusion and calibration pipelines that are robust to lighting variations.
  • Real-time Performance: The system achieves real-time performance (over 100 Hz), which is crucial for applications like SLAM and robotics.

Key Results

  • Pose Estimation Accuracy: The system achieves millimeter accuracy in translation and a few degrees of error in rotation compared to ground truth from a motion capture system.
  • ID Decoding Accuracy: The RKHS-based ID decoding achieves 99.7% accuracy.
  • False Positive Rejection: No false positives were detected in over 379 thousand LiDAR scans across the Google Cartographer and Honda H3D datasets.
  • Real-time Performance: The system operates at over 100 Hz, outperforming current LiDAR sensor frequencies.
  • Detection Range: For a 1.2-meter tag, the detection limit was observed at 16 meters with a 45-degree rotation using a 32-beam LiDAR.

Limitations and Future Work

  • Motion Compensation: The current algorithm assumes motion-compensated point clouds. Adapting the algorithm to handle motion distortion is a direction for future work.
  • Deep Learning Integration: The authors suggest that if datasets were available and labeled, deep learning architectures could potentially replace the current LiDARTag detection process, offering another avenue for future research.
  • Multi-sensor Fusion: The paper highlights the potential for fusing camera-based tag systems with LiDARTag for enhanced robustness and capabilities.
  • LiDAR Sensor Variability: The detection range and performance can be influenced by the specific LiDAR sensor used (e.g., number of beams, point density).

Summary updated

BibTeX
@article{huang2021lidartag,
  author = {Jiunn-Kai Huang and Shoutian Wang and Maani Ghaffari and Jessy W. Grizzle},
  title = {{LiDARTag: A Real-Time Fiducial Tag System for Point Clouds}},
  journal = {IEEE RA-L},
  year = {2021},
  doi = {10.1109/lra.2021.3070302},
}
Figure