Introduction
Predicting the future trajectories of road users—vehicles, cyclists, and pedestrians—is a fundamental challenge for autonomous driving systems. Traditional approaches often rely on "object tracks," which simplify agents into bounding boxes with associated velocity and heading data. While computationally efficient, this representation is inherently "lossy," discarding subtle but critical behavioral cues. For instance, a pedestrian's gaze or a cyclist's hand signals can signal intent long before a change in velocity is detected. LiMTR (LiDAR Motion Transformer) addresses this by integrating raw, local LiDAR point clouds directly into the prediction pipeline to capture these "micro-behaviors."
Methodology
LiMTR is built upon the Motion Transformer (MTR) architecture, enhanced with a specialized LiDAR Encoder. Unlike global scene processing methods that suffer from resolution loss through voxelization, LiMTR utilizes a PointNet-based encoder to process raw points directly.
The technical framework emphasizes three key areas:
- Local Contextualization: The encoder focuses exclusively on LiDAR points within an agent's bounding box (plus a 15% margin), allowing for high-resolution analysis of specific road users without global overhead.
- Feature Fusion: The model processes 512 points per agent, incorporating 3D coordinates and reflectivity (intensity). These are passed through 12-layer MLP blocks and integrated into the MTR’s attention mechanism.
- Temporal Integration: By utilizing up to 11 frames (one second) of LiDAR history, the model captures dynamic changes in pose and orientation over time.
Experimental Setup
The model was evaluated using the Waymo Open Dataset (WOD), which provides a massive-scale benchmark with 104,000 scenarios. WOD includes high-frequency tracking and raw sensor data, making it ideal for testing the integration of fine-grained LiDAR features in complex urban environments.
Results and Discussion
LiMTR demonstrates significant performance improvements over state-of-the-art baselines, particularly for vulnerable road users (VRUs).
Key findings include:
- Accuracy Gains: LiMTR achieved a 6.20% improvement in minimum Average Displacement Error (minADE) and a 1.58% boost in mean Average Precision (mAP) compared to the standard MTR baseline.
- VRU Sensitivity: The most pronounced gains were observed in pedestrians and cyclists. This validates the hypothesis that body language and posture—captured via LiDAR—are more predictive for human agents than for vehicles, which are largely constrained by physics and road geometry.
- Feature Importance: Ablation studies highlighted that LiDAR intensity is a more critical feature than elongation, likely due to its ability to distinguish surface materials. Furthermore, performance scaled with temporal depth, confirming that the evolution of an agent's pose is a vital predictor of intent.
Conclusion and Future Work
By localized processing of raw LiDAR data, LiMTR successfully bridges the gap between simplified tracking and high-fidelity sensor perception. While the addition of a 22M-parameter encoder introduces some computational overhead, the resulting gains in safety-critical VRU prediction are substantial. Future research will explore deeper end-to-end integration of perception and prediction and direct comparisons with voxel-based methods on specific intent-driven scenarios.
Summary updated