Introduction
Tracking objects in 3D using LIDAR is a fundamental challenge for autonomous systems. Unlike 2D images, LIDAR point clouds are sparse, unstructured, and computationally expensive to search. Traditional state-of-the-art methods typically rely on exhaustive searches—which are too slow for real-time applications—or Bayesian filters like Kalman and Particle filters to estimate object trajectories.
However, these filters often generate poor "proposals" because they rely on simple motion models rather than the actual geometric features of the sensor data. This frequently leads to tracking drift and failure. This paper addresses these limitations by introducing a method to generate a small, highly accurate set of 3D candidates without the massive overhead of searching the entire 3D space.
Proposed Architecture: The Double Siamese Network
The authors propose a Double Siamese Network architecture that decomposes the tracking task into two specialized stages: 2D proposal generation and 3D refinement.
1. 2D BEV Proposal Generation
The raw 3D point cloud is first projected into a 2D Bird’s Eye View (BEV) image. This representation is dense and structured, enabling the use of standard Convolutional Neural Networks (CNNs). A Region Proposal Network (RPN), built on a 2D Siamese backbone, quickly identifies a small set of likely object locations (candidates) based on visual features rather than just motion priors.
2. 3D Siamese Refinement and Shape Completion
The 2D proposals are converted back into 3D bounding boxes, and the points within are processed by a 3D Siamese network (based on PointNet). This network calculates a similarity score between the candidates and the original object template. To enhance robustness, the network is also trained on a Shape Completion task. Since LIDAR only captures the side of an object facing the sensor, the network learns to reconstruct the full 3D shape, which helps it better distinguish the target from background clutter.
Key Insights
- Dimensionality Reduction: Searching in 2D (BEV) is significantly more efficient than 3D. While BEV loses some detail, it retains enough information to provide high-quality proposals, allowing the 3D network to focus solely on fine-grained discrimination.
- Visual Features vs. Motion Priors: Unlike Kalman filters that guess where an object should be, the RPN looks at what the environment actually looks like, allowing the system to recover from abrupt movements.
- Latent Feature Regularization: By forcing the network to "imagine" the full 3D shape of a vehicle from a partial view, the learned latent features become more descriptive and effective for tracking.
Experimental Results
The method was evaluated on the KITTI dataset, focusing on Cars, Cyclists, and Pedestrians. The results demonstrate a significant performance leap over traditional filtering baselines.
On the Car class, the method achieved a 12% boost in Success (AUC of IoU) and an 18% boost in Precision (AUC of center distance). Crucially, the RPN-based proposals are so accurate that using just 16 candidates yields better results than using 147 candidates with traditional filtering methods, suggesting a potential 15x speedup in the similarity selection phase.
Limitations and Future Work
While highly effective for rigid objects like cars, the "shape completion" benefit was less pronounced for deformable objects like pedestrians. Because humans change shape significantly while walking, a simple rigid-body assumption is insufficient. Future work may involve incorporating human body pose encoding. Additionally, a "selection gap" remains between the "Best Proposal" and the final tracked result, indicating that the 3D Siamese discriminator can be further refined to better select the optimal candidate from the RPN output.
Summary updated