Trajectory Forecasting as a Discrete Classification Task

22 concepts18 papers1 proposal

Imagine an autonomous vehicle approaching a fork in the road. The driver ahead could turn left or turn right with equal probability. If a trajectory forecasting network uses standard continuous regression—minimizing Mean Squared Error (MSE) to predict the exact future coordinates of that vehicle—what does it do? It averages the two distinct possibilities and predicts that the driver will plow straight ahead into the concrete divider.

This is the fundamental limitation of continuous regression in behavior modeling: it struggles with multi-modality. When an agent faces mutually exclusive choices, regression models often suffer from mode collapse or output physically impossible "averaged" paths.

To solve this, a powerful paradigm shift has emerged across robotics, computer vision, and traffic modeling. Instead of asking "what are the exact continuous coordinates of the agent over time?", researchers reframe the problem as a discrete classification task: "which of these predefined paths, regions, or maneuvers is the agent most likely to choose?"

Discretizing Space and Time

The simplest way to transition from regression to classification is to partition the continuous world into a discrete set of bins or tokens.

Continuous Space (Regression)          Quantized Grid (Classification)
          (x, y)                                 [ ] [X] [ ]
            |                                    [ ] [ ] [ ]
            v                                    [ ] [ ] [ ]
    [Blurs Multi-Modality]               [Assigns Distinct Probabilities]

In early work, Eyjolfsdottir et al. demonstrated this by predicting probability distributions over discretized spatial bins to capture the non-deterministic, multi-modal behaviors of biological agents. Rather than outputting a single coordinate, the model outputs a probability distribution over the entire grid.

This spatial quantization strategy scales remarkably well to complex human environments. For instance, Giuliari et al. and Franco et al. showed that stripping away complex social interaction modules and simply quantizing continuous coordinate spaces into discrete spatial clusters allows standard Transformer networks to forecast pedestrian trajectories with high accuracy.

Beyond static grids, motion can also be discretized into sequential tokens or templates. Luo et al. quantized continuous video motion into "atomic 3D flows," transforming long-term video forecasting into a stable classification task. Taking this concept to its logical conclusion, Philion et al. introduced Trajeglish, a framework that treats traffic modeling as a next-token prediction task over a vocabulary of motion templates, effectively modeling vehicle and pedestrian trajectories exactly like natural language. Even in goal-conditioned reinforcement learning, Hejna et al. demonstrated that modeling the time-to-goal as a discrete probability distribution captures the multi-modal nature of offline datasets far better than scalar value regression.

Trajectory Anchors and Maneuver Menus

Instead of classifying motion frame-by-frame or cell-by-cell, an alternative approach is to classify across entire future trajectories or high-level intents simultaneously.

In CoverNet, Phan-Minh et al. framed trajectory prediction as a classification task over a "menu" of diverse, physically feasible candidate trajectories. Because the model selects from a predefined set of mathematically sound paths, it completely avoids the averaging effect and guarantees that every predicted path is dynamically possible for a vehicle to execute. Similarly, Chen et al. mapped continuous future paths to a finite set of basis trajectories, turning motion prediction into a multi-label classification task that maintains strict error bounds for real-time planning.

      [Turn Left Path]  --> Probability: 45%
    /
---*-- [Straight Path]   --> Probability: 5%
    \
      [Turn Right Path] --> Probability: 50%

Classification can also operate on high-level semantic behaviors rather than raw geometry:

  • Maneuver Classification: Liniger et al. designed a system that outputs probability distributions over explicit maneuvers like lane changes and turns, making the predictions highly interpretable for downstream planners.
  • Latent Decision Modes: Villaflor et al. leveraged discrete latent behavior modes as "anchors" to evaluate how other drivers react to an ego-vehicle's actions, showing that discrete modes provide clearer semantic distinctions than continuous latent variables.
  • Sparse Destination Ranking: Even when tracking data is highly intermittent, Luan et al. demonstrated that framing destination prediction as a binary classification problem over a set of candidate locations allows a model to robustly rank the likelihood of discrete potential destinations.

Why It Works: The Intuition of the Softmax Layer

The core reason discrete trajectory classification outperforms continuous regression in multi-modal settings lies in the mathematical behavior of the network's final layer.

A regression network typically outputs a point estimate by minimizing a distance metric like MSE. When forced to choose between two diverging paths, the mathematical minimum of the loss function is the geometric mean—the exact center between the two choices.

In contrast, a classification network utilizes a softmax layer over the discrete options. The softmax function does not calculate an average; it assigns independent likelihoods to each option. If an agent faces a fork in the road, the network can output a probability vector of [0.5, 0.0, 0.5], placing high confidence on the left and right options while assigning zero probability to the impossible middle ground.

As Jain et al. demonstrated with discrete residual flows, modeling environments as discrete spatial grids allows networks to represent arbitrary, non-Gaussian probability distributions. This bypasses the rigid mathematical constraints of parametric mixture models (like Gaussian Mixture Models), which frequently suffer from numerical instability during training.

Furthermore, this approach eliminates the need for computationally expensive generative sampling. Instead of running a diffusion model or a GAN hundreds of times to sample potential futures, the authors of TimePrism (Dai et al.) showed that a model can directly map historical data to a finite set of discrete future scenarios and their explicit probabilities. This direct mapping achieves state-of-the-art forecasting accuracy while operating up to five orders of magnitude faster than continuous sampling methods.

Structural Coherence and Interpretability

Beyond handling multi-modality, discretization provides significant structural advantages. When initial inputs are highly ambiguous, a standard model's predictions can become blurred and structurally incoherent. To combat this, Hayashi et al. factorized trajectory prediction into two distinct parts: a categorical distribution over potential classes, and class-conditional probability distributions for the next state. This factorization ensures that even if the overall future is uncertain, each individual trajectory hypothesis remains structurally sharp and self-consistent.

Finally, converting continuous paths into discrete categories makes these complex neural networks auditable. As Tas et al. demonstrated, mapping continuous trajectories into discrete categorical bins representing specific physical behaviors (such as speed thresholds or directional shifts) allows researchers to apply standard classification-based interpretability methods directly to motion transformers, exposing the underlying control vectors that drive the model's decisions.

Key Foundational Work

Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction (2019) — contributed 2 concept elements to this direction.
Learning Recurrent Representations for Hierarchical Behavior Modeling (2017) — contributed 1 concept elements to this direction.
Unsupervised Learning of Long-Term Motion Dynamics for Videos (2017) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_discrete_trajectory_classification_f672eb23,
  title = {discrete trajectory classification},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/discrete-trajectory-classification/rdr_f672eb23871646fea5e04d7206d3e6e8},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_discrete_trajectory_classification_f672eb23_v1,
  title = {discrete trajectory classification},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/discrete-trajectory-classification/rdr_f672eb23871646fea5e04d7206d3e6e8/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake