Autoregressive Meta-Actions for Unified Controllable Trajectory Generation

Jianbo Zhao, Taiyu Ban, Xiyang Wang, Qibin Zhou, Hangning Zhou, Zhihao Liu, Mu Yang, Lei Liu, Bin LiarXiv 2025 · DOI · arXiv

Introduction: The Semantic Misalignment Problem

In autonomous driving, "meta-actions" are high-level semantic commands such as lane change, turn left, or keep lane. A trajectory generator should ideally translate these commands into smooth, realistic paths. However, existing frameworks often suffer from semantic misalignment. Traditionally, a single meta-action label is assigned to a fixed, long time interval (e.g., a 5-second window). Because driving is fluid, a car might spend the first few seconds "keeping lane" before actually initiating a "lane change." Forcing a model to associate "lane change" with straight driving creates a noisy trajectory space, hindering controllability and responsiveness.

Comparison of traditional long-interval labels vs. the proposed frame-level meta-actions, illustrating how a single interval can contain multiple distinct behaviors.

Methodology: The ARMA Framework

The authors propose Autoregressive Meta-Actions (ARMA), a unified formulation that treats meta-actions and trajectory states with the same temporal granularity.

Frame-Level Decomposition and Sequential Interplay

Instead of one label for a long window, every single frame (unit time interval) receives its own meta-action label. The model operates in two steps at each time step $\tau$:

  1. Meta-Action Prediction: It predicts the next frame's meta-action based on history.
  2. Conditioned Generation: It generates the next trajectory state (position, velocity, etc.) conditioned specifically on that predicted meta-action.

Staged Modular Training

To maintain flexibility and stability, ARMA employs a "plug-and-play" training strategy:

  • Stage 1: Train a "Foundation Model" on raw trajectory data to learn basic motion dynamics and environment interactions.
  • Stage 2: Freeze the foundation model and train two small additional modules—a Meta-Action Prediction module and a Meta-Action Injection module. This allows for adding new types of decisions without retraining the entire motion engine.
The ARMA architecture featuring the Foundation Model, Meta-Action Prediction, and Injection modules, highlighting the modular training approach.

Kinematic-Guided Representation

Rather than predicting raw $(x, y)$ coordinates, the model predicts control actions (acceleration and yaw rate) using a Constant Turn Rate and Acceleration (CTRA) model, ensuring the output is physically feasible.

Key Insights

  • The "Privacy" of Intent: A critical architectural choice is the timing of meta-action injection. By injecting intent after processing the environment but before history aggregation, the model prevents "leaking" private intent to other agents through attention mechanisms, mimicking real-world driving.
  • Task Unification: By moving to frame-level labels, the paper achieves Structural Consistency (consistent input/output shapes) and Semantic Consistency (consistent function interpretation over time), reducing modeling complexity.
  • Decoupling "How" from "What": Freezing the foundation model proves that learning basic motion can be decoupled from learning to follow high-level commands, making the system easier to update.

Experimental Results

The ARMA framework was evaluated on the Waymo Motion Dataset v1.2, augmented with heuristic frame-level annotations for actions like Stationary, Keep Lane, Lane Change, Turn, and U-Turn.

  • Decision-Following Accuracy: ARMA achieved a Mean Average Precision (mAP) of 0.718, significantly outperforming regression-based baselines (0.567).
  • Qualitative Performance: In "Left Lane Change" scenarios, ARMA successfully executed maneuvers in 9 out of 10 samples, whereas traditional regression baselines failed entirely (0/10 success).
  • Efficiency: The staged fine-tuning approach provided a 2.25x training speedup compared to training the entire system from scratch.
  • Realism: ARMA maintained high scores on the Waymo SimAgents benchmark, showing that adding controllability does not degrade trajectory naturalness.

Conclusion and Future Work

ARMA effectively addresses semantic misalignment through frame-level autoregressive modeling. While successful, the current discrete action space and heuristic labeling present opportunities for refinement. Future work may explore expanded action vocabularies and end-to-end learned labeling to further sharpen decision boundaries in complex driving scenarios.

Summary updated

BibTeX
@misc{zhao2025autoregressive,
  author = {Jianbo Zhao and Taiyu Ban and Xiyang Wang and Qibin Zhou and Hangning Zhou and Zhihao Liu and Mu Yang and Lei Liu and Bin Li},
  title = {{Autoregressive Meta-Actions for Unified Controllable Trajectory Generation}},
  year = {2025},
  eprint = {2505.23612},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.23612},
}
Figure