Abstract: Most existing perception systems rely on sensory data acquired from cameras, which perform poorly in low light and adverse weather conditions. To resolve this limitation, we have witnessed advanced LiDAR sensors become popular in perception tasks in autonomous driving applications. Nevertheless, their usage in traffic monitoring systems is less ubiquitous. We identify two significant obstacles in cost-effectively and efficiently developing such a LiDAR-based traffic monitoring system: (i) public LiDAR datasets are insufficient for supporting perception tasks in infrastructure systems, and (ii) 3D annotations on LiDAR point clouds are time-consuming and expensive. To fill this gap, we present an efficient semi-automated annotation tool that automatically annotates LiDAR sequences with tracking algorithms while offering a fully annotated infrastructure LiDAR dataset — FLORIDA (Florida LiDAR-based Object Recognition and Intelligent Data Annotation) — which will be made publicly available. Our advanced annotation tool seamlessly integrates multi-object tracking (MOT), single-object tracking (SOT), and suitable trajectory post-processing techniques. Specifically, we introduce a human-in-the-loop schema in which annotators recursively fix and refine annotations imperfectly predicted by our tool and incrementally add them to the training dataset to obtain better SOT and MOT models. By repeating the process, we significantly increase the overall annotation speed by three to four times and obtain better qualitative annotations than a state-of-the-art annotation tool. The human annotation experiments verify the effectiveness of our annotation tool. In addition, we provide detailed statistics and object detection evaluation results for our dataset in serving as a benchmark for perception tasks at traffic intersections.
Introduction
While LiDAR has revolutionized autonomous vehicle perception, its application in "smart infrastructure" (like traffic light monitoring) lags behind. There are two primary bottlenecks preventing widespread adoption. First, Data Scarcity: Most public LiDAR datasets, such as KITTI or Waymo, are collected from moving vehicles. Infrastructure LiDAR has a different perspective—top-down and stationary—making models trained on onboard data perform poorly. Second, the "Annotation Tax": Labeling 3D point clouds is grueling. A single bounding box has seven degrees of freedom (x, y, z, length, width, height, and heading). For context, a recent pedestrian dataset required 960 person-hours to label 219K boxes. Existing tools often fail when point clouds are sparse or objects are partially occluded.
Methodology: A Human-in-the-Loop Pipeline
The proposed system introduces a semi-automated 3D annotation pipeline that speeds up infrastructure LiDAR labeling by 3–4x using a human-in-the-loop approach. The tool integrates two types of deep-learning-based tracking to minimize manual effort:
- Single-Object Tracking (SOT): When an annotator clicks an object, the tool uses the $M^2$-track algorithm to propagate that box across subsequent frames. This is particularly effective for infrastructure because the stationary sensor makes motion cues highly reliable.
- Multi-Object Tracking (MOT): The tool uses SimpleTrack (built on a CenterPoint detector) to auto-label entire sequences.
- Incremental Learning: This is the core "loop." Annotators fix the imperfect predictions from the MOT/SOT models. These corrected labels are then fed back into the training set to retrain the models, making the tool progressively smarter and faster as the project continues.
To handle the sparsity of pedestrian data, the tool includes trajectory smoothing using cubic splines and an orientation auto-correction feature that aligns an object's heading with its path of travel.
The FLORIDA Dataset
Alongside the tool, the authors release FLORIDA (Florida LiDAR-based Object Recognition and Intelligent Data Annotation), a large-scale infrastructure LiDAR dataset designed to bridge the gap between onboard autonomous driving data and stationary traffic monitoring needs.
- Scope: 11 sequences collected at a busy intersection in Gainesville, FL, using a 32-channel Velodyne LiDAR.
- Content: Over 249,000 annotated instances across six categories: Vehicle, Pedestrian, Cyclist, Motorcycle, Bus, and Truck.
- Significance: Unlike previous infrastructure datasets (like Ko-PER or LUMPI), FLORIDA includes crowded pedestrian scenarios and is released with full labels and object detection baselines.
Key Insights and Results
The Stationary Advantage
In infrastructure LiDAR, the background never moves. The authors exploit this by using motion-centric tracking paradigms ($M^2$-track), which focus on relative motion between frames rather than just appearance. Furthermore, the MOT model acts as a "second pair of eyes," flagging stationary objects that human annotators might naturally overlook while searching for motion.
Efficiency and Accuracy Gains
The tool nearly quadrupled the annotation speed compared to SUSTechPOINTS (a popular open-source tool). For pedestrians, the annotation rate jumped from ~27 to ~72 boxes per minute. Additionally, the F1-score for untrained annotators improved by 28.8% for pedestrians, proving that the tool makes high-quality labeling accessible to non-experts. The authors established 3D Average Precision (AP) baselines for the FLORIDA dataset, achieving 90.66% for vehicles and 87.44% for pedestrians.
Conclusion and Future Work
While the current system relies solely on point clouds, future work will involve integrating video camera feeds (sensor fusion) to provide annotators with more visual context. The authors also plan to explore how well models trained on the FLORIDA dataset generalize to entirely different intersections with varying mounting heights and angles. Despite current challenges with extremely crowded scenes, the semi-automated approach significantly lowers the barrier to creating high-quality infrastructure datasets.
Summary updated