Controllable Generative Trajectory Prediction via Weak Preference Alignment

Yongxi Cao, Julian Frederik Schumann, Jens Kober, Joni Pajarinen, Arkady ZgonnikovSubmission to ICLR 2026 · Rejected

TL;DR

PrefCVAE augments conditional variational autoencoders (CVAEs) with weak preference alignment to give autonomous vehicle planners direct, continuous control over predicted agent trajectories. By training the model to align paired trajectory rollouts with simple ordinal preferences (e.g., faster vs. slower), it enables fine-grained semantic control (such as driving speed) without sacrificing standard prediction accuracy.

What Problem Does This Paper Address?

In autonomous driving, motion forecasting must account for the uncertain and multimodal behavior of surrounding road users. While deep generative models like CVAEs effectively generate diverse trajectories, their latent spaces are implicit, "black-box" representations.

Current models default to the single most likely path or produce randomly diversified futures. However, safe motion planning requires controllably diverse predictions: an ego-vehicle needs to evaluate specific "what-if" scenarios (e.g., "What if the oncoming vehicle drives aggressively versus conservatively?"). Existing methods either rely on expensive discrete modes, rejection sampling, or require fully supervised ground-truth latent labels that are rarely available.

Methods

The authors introduce PrefCVAE, which factorizes the latent space into interpretable semantic factors ($z_S$) and implicit noise factors ($z_I$):

  • Weak Preference Labelling: Instead of requiring exact numerical ground truth for latent attributes, the framework samples pairs of semantic latents ($z_0, z_1$) and uses the CVAE decoder to roll out candidate trajectories $(\hat{y}_0, \hat{y}_1)$. An oracle metric (e.g., average velocity) calculates the relative ranking between the two rollouts via a smooth, differentiable Sigmoid function.
  • Preference Alignment Loss: Inspired by preference optimization in RLHF, a cross-entropy loss ($\mathcal{L}_{\text{pref}}$) penalizes the network if the decoder's generated trajectory attributes do not match the ordinal relationship of the sampled latents.
  • Beta-Distributed Latents: Standard Gaussian priors are replaced with bounded Beta distributions ($\alpha, \beta > 1$ parameterized via ELUs), mapping the semantic attributes cleanly to a bounded interval $[0, 1]$.
  • Base Architecture: The method is evaluated on a modified version of AgentFormer (termed $\beta$-AgentFormer), an attention-based multi-agent trajectory prediction model.
Sample Latents (z0 < z1) ──► CVAE Decoder ──► Rollout Trajectories (ŷ0, ŷ1)
                                                    │
                                           Oracle Metric m(·)
                                                    ▼
                     Preference Loss ◄── Compare (m(ŷ0) vs. m(ŷ1))

Datasets

  • nuScenes: Evaluated on the standard nuScenes vehicle trajectory forecasting benchmark (predicting 6 seconds / 12 future frames given 1.5 seconds / 4 history frames). The test split includes 138 scenes, 3,076 minibatches, and 9,041 agents.
  • No new datasets or external benchmarks were introduced.

Key Insights and Ideas

  • Weak ordinal supervision is sufficient: The model does not need precise numerical labels for latent factors. Simply asking, "Is rollout A faster than rollout B?" is enough to structure continuous, monotonic latent dimensions.
  • Dropping preferences prevents overfitting: Surprisingly, training on only 25% of the preference pairs ($\nu = 0.25$) achieved better consistency and lower violation rates than using 100% ($\nu = 1.0$), as partial sampling prevents the model from overfitting to rollout noise.
  • Decoupled multi-agent manipulation: Even when all agents are modeled jointly in a scene-level Transformer, altering the semantic latent assigned to a single vehicle modifies only that target vehicle's behavior monotonically, leaving neighboring agents unaffected.
  • Model-agnostic representations: Encoders and decoders trained under different preference weights ($\lambda$) learned aligned semantic latent representations that transferred across configurations.

Key Results

Using average velocity as the target continuous semantic attribute:

  • Monotonic Controllability: Traversing the latent code from $0.1$ to $0.9$ consistently scales predicted vehicle speed across the test set (velocity range spans from $1.87$ to $4.38\text{ m/s}$ at $\nu=0.25$, and $1.63$ to $4.86\text{ m/s}$ at $\nu=1.0$). In contrast, the baseline model remains essentially static ($3.16$ to $3.17\text{ m/s}$) with non-monotonic outputs.
  • Preserved Accuracy: Controllability does not hurt baseline trajectory accuracy. PrefCVAE ($\nu = 0.25$) achieves $\text{minADE}_5 = 2.66\text{ m}$ and $\text{minFDE}_5 = 5.76\text{ m}$, virtually identical to the unconstrained baseline ($2.62\text{ m}$ and $5.62\text{ m}$).
  • Encoder Precision: The posterior encoder concentrates around ground-truth values. Mode deviation drops sharply from $0.1660$ (baseline) to $0.0090$ ($\nu = 0.25$), and log-likelihood at the mode improves from $3.19$ to $13.23$.
  • Low Violation Rate (VR): For $\nu \ge 0.25$ and $\lambda = 32$, the agent-wise violation rate drops to $0.32\%$ or lower.

Limitations and Future Work

  • Single Synthetic Semantic Factor: The method was validated solely using an analytical, differentiable oracle metric (average speed).
  • Extension to Human Feedback: Incorporating non-differentiable or subjective human judgments will require discrete gradient estimators (e.g., Gumbel-Softmax).
  • Multi-Factor Disentanglement: The paper only tests a single 1D semantic factor. Scaling to multiple simultaneous attributes (e.g., velocity alongside Social Value Orientation or lane-change intent) remains challenging due to latent correlations and dimension entanglement.

Summary updated

Abstract

Deep generative models such as conditional variational autoencoders (CVAEs) have shown great promise for predicting trajectories of surrounding agents in autonomous vehicle planning. State-of-the-art models have achieved remarkable accuracy in such prediction tasks. Besides accuracy, diversity is also crucial for safe planning because human behaviors are inherently uncertain and multimodal. However, existing methods generally lack a scheme to generate controllably diverse trajectories, which is arguably more useful than randomly diversified trajectories, to the end of safe planning. To address this, we propose PrefCVAE, an augmented CVAE framework that uses weakly labeled preference pairs to imbue latent variables with semantic attributes. Using average velocity as an example attribute, we demonstrate that PrefCVAE enables controllable, semantically meaningful predictions without degrading baseline accuracy. Our results show the effectiveness of preference supervision as a cost-effective way to enhance sampling-based generative models.
BibTeX
@misc{cao2025controllable,
  author = {Yongxi Cao and Julian Frederik Schumann and Jens Kober and Joni Pajarinen and Arkady Zgonnikov},
  title = {{Controllable Generative Trajectory Prediction via Weak Preference Alignment}},
  year = {2025},
  eprint = {2510.10731},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.10731},
}
Figure