From Tokens to Behaviors: Rethinking Generative Recommendation for User Trajectory Prediction

Jiaqi Wang, Hanchen Yang, Hengyu Zhang, Hongxu Ma, Wenqi Rao, Hongyang Wang, Zhongbo Sun, Renliang Qian, Weixin Chen, Wengen Li, Jihong Guan, Shuigeng Zhou, Qingpeng Cai, Peng Jiang, Hong ChengarXiv 2026 · arXiv

TL;DR

TraGR (Behavior-aware Generative Recommendation) shifts the focus of recommender systems from predicting a single "next item" to forecasting entire sequences of future user actions. By decoupling item token generation from behavioral transitions and introducing "probability feedback" to handle predictive uncertainty, TraGR significantly outperforms existing methods in accuracy and inference speed across long-term prediction horizons.

What Problem Does This Paper Address?

Most modern recommendation engines are optimized for next-item prediction: given a user's history, what is the very next thing they will click or buy? However, user needs evolve in trajectories. A user buying running shoes might soon need sports socks, followed by a water bottle. Predicting this entire User Behavior Trajectory (UBT) is vital for proactive service and long-term engagement.

Current Generative Recommendation (GR) models, which represent items as sequences of "Semantic IDs" (SIDs), struggle with three main issues:

  1. Granularity Mismatch: They often use serial token-by-token decoding, which mixes up the process of "building" an item ID with the process of "transitioning" to the next behavior.
  2. Neglect of Uncertainty: When generating a trajectory, models usually feed only the single "best" predicted item back into the system to predict the next one. This ignores other highly probable alternatives, leading to "cascading errors" if the first choice was slightly off.
  3. Training-Inference Gap: Models are typically trained using "teacher forcing" (always seeing the ground truth), but during actual use, they must rely on their own previous (and potentially incorrect) predictions.
Fig. 1. Generative recommendation for user trajectory prediction. (a) Next item candidates are alternatives for one interaction, whereas a trajectory contains items at successive future positions. (b) With socks selected, item-only feedback may predict more socks, while a probability summary retains support for running shoes and guides the next prediction. (c) Model-generated trajectories require relation-aware learning to address the training–inference gap.

Methods

The authors propose TraGR, a framework designed to align generation, feedback, and learning with behavior-level transitions.

  • Behavior-Level Architecture: Instead of decoding SID tokens one by one, TraGR predicts all tokens for an item in parallel. The "behavioral state" (the model's memory of the user) only updates once per item interaction, rather than once per token.
  • Probability Feedback: To retain predictive uncertainty, TraGR doesn't just feed back the selected item. It creates a "feedback embedding" that blends the selected item with a probability-weighted summary of other plausible candidates. This allows the model to "remember" that while it picked socks, running shoes were also a strong possibility, which might better explain a subsequent water bottle purchase.
  • Relation-Aware Trajectory Learning: The model is trained in two stages. Stage I uses standard supervised learning. Stage II uses a reinforcement learning-style approach where the model generates trajectories based on its own predictions. These are rewarded based on position accuracy, collaborative transition signals (common patterns seen across all users), and "later credit"—rewarding an early choice if it eventually leads to a correct prediction several steps later.
Overview of the TraGR architecture and two-stage training process.

Datasets

The model was evaluated on four diverse public datasets:

  • Amazon Beauty & Cell Phones: Shorter trajectories (predicting 3 future items).
  • MovieLens-1M (ML-1M) & Last-FM: Longer trajectories (predicting 5 future items).

The authors used a chronological "leave-out" evaluation, ensuring the model was tested on the most recent interactions it hadn't seen during training.

Key Insights and Ideas

  • Parallelism is Key: Moving from serial token decoding to parallel SID prediction isn't just faster; it separates the "what is this item" logic from the "what happens next" logic, which is more representative of how users actually behave.
  • Soft Feedback Prevents Drift: The "probability feedback" mechanism acts as a safety net. By incorporating information about alternative items, the model is less likely to get stuck in a "feedback loop" of its own errors during long-term trajectory generation.
  • Collaborative Rewards: By rewarding transitions that are common in the general population (collaborative signals), the model learns to bridge gaps when a specific user's history is sparse.

Key Results

TraGR consistently outperformed 13 state-of-the-art baselines (including diffusion-based models like LPDO and autoregressive models like GCB) across all metrics.

  • Accuracy: On the SeqMatch@50 metric (which measures if the model got the entire trajectory right), TraGR showed relative improvements of up to 75.65% on the Cell Phones dataset.
  • Horizon Stability: As shown in Figure 6, while all models' performance drops as they try to predict further into the future ($K=10$), TraGR maintains a significantly higher accuracy floor compared to competitors.
  • Efficiency: Thanks to parallel decoding, TraGR is substantially faster than beam-search-based generative models. On the ML-1M dataset, its inference time was roughly 23x faster than the GCB baseline.
Performance of TraGR across different prediction horizons K.

Limitations and Future Work

The primary limitation acknowledged is computational scaling. The probability feedback mechanism currently requires a summation over the entire item catalog. While manageable for the datasets tested (up to ~12k items), this could become a bottleneck for industrial catalogs with millions of items.

The authors suggest that future work should focus on efficient candidate selection (e.g., using approximate nearest neighbor search) to aggregate feedback more scalably without losing the benefits of uncertainty modeling.

Summary updated

Abstract

User behavior trajectory prediction (UBTP) aims to capture how a user's needs evolve over time by predicting the sequence of future interactions from that user's interaction history. Recent generative recommendation (GR) methods address this task through autoregressive generation of semantic IDs (SIDs). However, serial SID decoding couples token generation with behavior state transitions and requires multiple dependent steps per interaction. Discrete item feedback overlooks predictive uncertainty, and teacher-forced training leaves a training-inference gap in trajectory generation. To address these limitations, we propose TraGR, a behavior-aware generative recommendation framework that aligns generation, feedback, and learning with successive user behaviors. Its behavior-level architecture predicts SID tokens in parallel within each interaction and models behavioral transitions through recurrent state updates, with joint SID and item-level supervision. Probability feedback combines the selected behavior with a probability-weighted summary of alternatives to guide subsequent predictions. Relation-aware trajectory learning incorporates collaborative transition signals and later credit assignment into training on model-generated trajectories. Experiments on four public datasets demonstrate that TraGR consistently outperforms state-of-the-art baselines across different prediction horizons. Code: https://github.com/CalmaQi/GR4UBT.
BibTeX
@misc{wang2026from,
  author = {Jiaqi Wang and Hanchen Yang and Hengyu Zhang and Hongxu Ma and Wenqi Rao and Hongyang Wang and Zhongbo Sun and Renliang Qian and Weixin Chen and Wengen Li and Jihong Guan and Shuigeng Zhou and Qingpeng Cai and Peng Jiang and Hong Cheng},
  title = {{From Tokens to Behaviors: Rethinking Generative Recommendation for User Trajectory Prediction}},
  year = {2026},
  eprint = {2611.16159},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16159},
}
Figure