Query-Centric Trajectory Prediction

Zikang Zhou, Jianping Wang, Yung-Hui Li, Yu-Kai Huang

Abstract

Predicting the future trajectories of surrounding agents is a fundamental challenge for autonomous driving systems. Traditional agent-centric models suffer from significant computational redundancy due to the need for frequent re-normalization as the observation window shifts. This paper presents QCNet (Query-Centric Network), a novel framework that employs a query-centric paradigm to enable efficient, streaming scene encoding. By utilizing local spacetime coordinate systems, QCNet produces roto-translation invariant features that can be cached and reused across time steps. Furthermore, a hybrid query-based decoder combines recurrent anchor-free proposals with anchor-based refinement to handle long-term multimodality. QCNet achieves state-of-the-art results on the Argoverse 1 and 2 benchmarks, demonstrating a 6x speedup in inference latency through its streaming architecture.

Introduction

Safe autonomous navigation requires accurately forecasting the future paths of vehicles and pedestrians. However, current state-of-the-art models often struggle with two primary issues: computational redundancy and the complexity of long-term multimodality. Most modern architectures are "agent-centric," meaning they normalize the entire scene relative to the target agent’s current position and heading. As the vehicle moves, this reference point changes, forcing the model to re-encode the entire scene history from scratch at every frame. Additionally, as the prediction horizon extends to six seconds or more, the exponential increase in possible maneuvers makes stable, multimodal prediction difficult for standard anchor-free or handcrafted anchor-based methods.

Methodology

Query-Centric Scene Encoding

QCNet addresses computational redundancy by shifting from a global or agent-centric coordinate system to a local spacetime coordinate system for every individual element. This includes every agent state at every time step and every map polygon.

Illustration of query-centric local coordinate systems across time steps.

As shown in Figure 1, because features are learned within these local frames, they are inherently roto-translation invariant. An agent's encoding at time $t-1$ remains valid even when the observation window moves to time $t$. This invariance allows for streaming inference: the system can cache previous encodings and only process new data frames as they arrive. This approach effectively "unlocks" the ability to reuse over 90% of the computation from previous time steps, drastically reducing the overhead of processing long histories.

Hybrid Query-Based Decoding

To manage the uncertainty of long-term trajectories, QCNet employs a two-stage hybrid decoder.

Architecture of the hybrid query-based decoder showing the proposal and refinement stages.
  1. Recurrent Anchor-Free Proposals: This module generates initial trajectory guesses by breaking the long-term prediction into smaller recurrent chunks. This allows the model to dynamically "attend" to relevant map features as the predicted agent moves through the environment.
  2. Anchor-Based Refinement: The initial proposals serve as dynamic anchors for a second refinement module. This stage refines the trajectories and assigns probability scores, combining the flexibility of anchor-free methods with the stability and spatial priors typical of anchor-based approaches.

Evaluation and Results

Datasets

QCNet was evaluated on the Argoverse 1 and Argoverse 2 motion forecasting benchmarks. Argoverse 2 is particularly challenging, featuring a 6-second prediction horizon and high data diversity across six different cities.

Performance and Efficiency

At the time of submission, QCNet ranked 1st place on both leaderboards. On Argoverse 2, it achieved a minFDE6 of 1.19 meters, outperforming established models like MTR and GANet.

The most significant impact of the query-centric design is seen in inference efficiency. In dense urban scenarios with up to 190 agents, the streaming capability reduced inference latency from 82ms (without caching) to just 13ms (with caching). This nearly 6x speedup makes the model highly suitable for real-time deployment on autonomous vehicle hardware.

Conclusion

QCNet demonstrates that query-centric encoding is a powerful paradigm for motion forecasting, offering a "free lunch" in terms of computational efficiency through feature reuse. While the factorized attention mechanism (separating temporal, social, and agent-map interactions) remains a potential bottleneck in extremely crowded environments, the overall architecture sets a new standard for accuracy and speed. Future work will focus on integrating these predictions more tightly with downstream planning modules to ensure that forecasted trajectories directly enhance vehicle safety and decision-making.

Summary updated

Abstract

Predicting the future trajectories of surrounding agents is essential for autonomous vehicles to operate safely. This paper presents QCNet, a modeling framework toward pushing the boundaries of trajectory prediction. First, we identify that the agent-centric modeling scheme used by existing approaches requires re-normalizing and re-encoding the input whenever the observation window slides forward, leading to redundant computations during online prediction. To overcome this limitation and achieve faster inference, we introduce a query-centric paradigm for scene encoding, which enables the reuse of past computations by learning representations independent of the global spacetime coordinate system. Sharing the invariant scene features among all target agents further allows the parallelism of multi-agent trajectory decoding. Second, even given rich encodings of the scene, existing decoding strategies struggle to capture the multimodality inherent in agents' future behavior, especially when the prediction horizon is long. To tackle this challenge, we first employ anchor-free queries to generate trajectory proposals in a recurrent fashion, which allows the model to utilize different scene contexts when decoding waypoints at different horizons. A refinement module then takes the trajectory proposals as anchors and leverages anchor-based queries to refine the trajectories further. By supplying adaptive and high-quality anchors to the refinement module, our query-based decoder can better deal with the multimodality in the output of trajectory prediction. Our approach ranks 1 st on Argoverse 1 and Argoverse 2 motion forecasting benchmarks, outperforming all methods on all main metrics by a large margin. Meanwhile, our model can achieve streaming scene encoding and parallel multi-agent decoding thanks to the query-centric design ethos.
BibTeX
@inproceedings{zhou2023query,
  author = {Zikang Zhou and Jianping Wang and Yung-Hui Li and Yu-Kai Huang},
  title = {{Query-Centric Trajectory Prediction}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.01713},
}
Figure