Trajectory Generation in New Environments from Past Experiences

Weiming Zhi, Tin Lai, Lionel Ott, Fabio RamosIROS 2021 · OpenReview

TL;DR

This paper introduces OCTNet, a novel framework for generating realistic human-like motion trajectories in new, unseen environments. OCTNet achieves this by learning to generalize motion patterns from past experiences in different environments, conditioned on the static layout (occupancy map) of the new environment. This allows autonomous systems to anticipate movement in unfamiliar spaces, a crucial step towards safer and more effective human-robot interaction.

What Problem Does This Paper Address?

Autonomous systems, like self-driving cars or service robots, need to navigate and interact safely within dynamic environments. A key challenge is predicting the future movements of other dynamic agents (pedestrians, vehicles). While many methods can predict trajectories based on past observations within a specific environment, they often fail to generalize to new environments where no prior motion data exists. This is problematic because motion patterns are heavily influenced by the environment's layout (e.g., corridors, rooms, open spaces). Simply extrapolating past motion without considering the new environment's geometry leads to unrealistic or unsafe predictions. The authors aim to bridge this gap by developing a system that can learn general motion principles and apply them to novel environments, using only the environment's static occupancy map as context.

Methods

The core of OCTNet is a probabilistic generative model that learns to map an environment's occupancy map to a distribution of possible trajectories. Here's a breakdown of their approach:

  • Occupancy Map Encoding: Each environment's occupancy map is first encoded into a feature vector. This is done non-parametrically by calculating a similarity between the queried map and all other maps in the training dataset. This similarity is based on a distance substitute kernel derived from the Hausdorff distance, effectively capturing how similar the layout of the new environment is to previously seen ones.
  • Trajectory Embedding: Observed discrete trajectories (sequences of waypoints) are represented as continuous functions. This is achieved by finding the best-fit continuous trajectory to the discrete data using Kernel Ridge Regression (KRR), and then embedding this continuous trajectory into a fixed-size vector of weights. This vector captures the essential characteristics of the trajectory.
  • Mixture Density Network (MDN): A Mixture Density Network (MDN) is trained to learn the conditional distribution of these trajectory embedding vectors, given the encoded occupancy map feature vector. The MDN is crucial for capturing the multi-modal nature of motion, meaning it can predict multiple distinct types of plausible trajectories (e.g., going left vs. going right from a junction). They explore using either Normal or Laplace distributions as the components within the MDN, finding Laplace distributions to perform better.
  • Trajectory Generation: To generate a trajectory in a new environment, the system first encodes the new environment's occupancy map. This encoded vector is fed into the trained MDN to obtain parameters for the conditional distribution of trajectory embedding vectors. By sampling from this distribution, they obtain embedding vectors, which are then converted back into continuous trajectories.
  • Rejection Sampling: Since the model doesn't inherently enforce that trajectories avoid occupied spaces, a rejection sampling step is used. Generated trajectories are checked against the environment's occupancy map, and any trajectory that overlaps with occupied areas is discarded and resampled.

Datasets

  • Occ-Traj 120: The primary dataset used for training and evaluation. This simulated dataset contains 120 binary occupancy grid maps of indoor environments (rooms and corridors) along with associated simulated motion trajectories.
  • The authors note that, to their knowledge, there isn't a large-scale real-world dataset with both occupancy maps and observed trajectories for diverse environments, which is a limitation.

Key Insights and Ideas

  • Generalization through Similarity: The key insight is that by encoding environments based on their similarity to known environments, the model can generalize learned motion patterns. This is more powerful than simply learning within a single environment.
  • Continuous Trajectory Representation: Representing trajectories as continuous functions, rather than just discrete waypoints, allows for smoother generation and querying at arbitrary resolutions, which is beneficial for planning.
  • MDN for Multi-Modality: The use of MDNs is critical for capturing the inherent multi-modal nature of human motion. For example, at an intersection, a person can go straight, turn left, or turn right, and the MDN can learn to predict all these possibilities.
  • Laplace Distribution Advantage: Empirically, using Laplace distributions as the prior for the trajectory embedding vector components yielded better results than Normal distributions, suggesting that the tails of the Laplace distribution better capture the variability in motion patterns.

Key Results

  • OCTNet significantly outperforms other generative models (GANs and CVAEs) in generating trajectories, as measured by Minimum Trajectory Distance (MTD) using Hausdorff, Frechet, and Dynamic Time Warping (DTW) metrics. For instance, OCTNet achieved MTD scores of 1.86 (Hausdorff), 2.00 (Frechet), and 93.10 (DTW) with Laplace distributions, compared to GANs at 11.79, 16.66, and 1147.40, respectively.
  • The generated trajectories are shown to be complex and multi-modal, often exhibiting different groupings of paths within the same environment, mirroring human behavior.
  • OCTNet can generate plausible trajectories in environments where no prior motion data was observed, demonstrating its generalization capabilities.

Limitations and Future Work

  • Lack of Temporal Dynamics: The current OCTNet model is conditioned only on static occupancy maps. It does not explicitly incorporate temporal changes or dynamics of the environment itself (e.g., moving obstacles, changing lighting).
  • Rejection Sampling Inefficiency: While effective, rejection sampling can be inefficient, especially if the model frequently generates invalid trajectories.
  • Dataset Scarcity: The authors highlight the need for real-world datasets with diverse environments and observed trajectories to further validate and improve the model.
  • Future Directions: The authors suggest incorporating temporal changes into the framework and exploring more sophisticated methods for trajectory generation and validation. They also emphasize the need for real-world data collection.

Summary updated

Abstract

Being able to safely operate for extended periods of time in dynamic environments is a critical capability for autonomous systems. This generally involves the prediction and understanding of motion patterns of dynamic entities, such as vehicles and people, in the surroundings. Many motion prediction methods in the literature implicitly account for environmental factors by learning on observed motion in a fixed environment, and are designed to make predictions in the same environment. In this paper, we address the problem of generating likely motion trajectories for novel environments, represented as occupancy grid maps, where motion has not been observed. We introduce the Occupancy-Conditional Trajectory Network (OTNet) framework, capable of transferring the previously observed motion patterns in known environments to new environments. OTNet provides a functional representation for motion trajectories and utilises neural networks to learn occupancy-conditional distributions over the function parameters. We empirically demonstrate our method’s ability to generate complex multi-modal trajectory patterns in both simulated and real-world environments.
BibTeX
@inproceedings{zhi2021trajectory,
  author = {Weiming Zhi and Tin Lai and Lionel Ott and Fabio Ramos},
  title = {{Trajectory Generation in New Environments from Past Experiences}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=M4ZUv_V1i86},
}
Figure