PLMTrajRec: A Scalable and Generalizable Trajectory Recovery Method with Pre-trained Language Models

Tonglong Wei, Yan Lin, Youfang Lin, Shengnan Guo, Jilin Hu, Cong Gao, Huaiyu Wan, Wan, Huaiyu

Abstract

Spatiotemporal trajectory data is crucial for various applications. However, issues such as device malfunctions and network instability often cause sparse trajectories, leading to lost detailed movement information. Recovering the missing points in sparse trajectories to restore the detailed information is thus essential. Despite recent progress, several challenges remain: the lack of large-scale dense trajectory data, varying spatiotemporal correlations across sampling intervals, and the difficulty of extracting road conditions for missing points. To address these, we propose PLMTrajRec, which leverages the scalability of a pre-trained language model (PLM). It uses dual trajectory prompts—explicit natural language instructions and implicit area flow-guided prompts—to handle sampling intervals and road conditions. Experiments on two public datasets demonstrate its effectiveness, scalability, and generalization ability.

Introduction

Spatiotemporal trajectory data, often referred to as the "breadcrumbs" of GPS-enabled movement, is fundamental to modern urban planning and traffic management. However, real-world data collection frequently results in "sparse" trajectories (e.g., one point every 4 minutes) due to hardware limitations or network instability. Restoring these to a "dense" state (e.g., one point every 15 seconds) is critical for downstream analysis.

Existing trajectory recovery methods face three primary hurdles. First, data scarcity: training deep learning models from scratch requires vast amounts of high-quality dense data. Second, sampling inconsistency: models often fail to generalize when the testing sampling interval differs from the training interval. Third, dynamic road conditions: movement patterns are heavily influenced by traffic, yet road conditions at missing points are inherently unknown.

Methodology

PLMTrajRec addresses these challenges by repurposing Pre-trained Language Models (PLMs), such as BERT, for trajectory recovery. The framework treats the problem as a sequence-filling task, utilizing the PLM's inherent ability to capture complex sequential dependencies.

The overall architecture of the PLMTrajRec framework.

The architecture consists of three main components:

  1. Dual Trajectory Prompts:
  2. Explicit Prompts: Natural language "instructions" (e.g., "The sparse trajectory is sampled every four minutes...") allow the PLM to adjust its reasoning based on the specific sampling rate.
  3. Implicit Prompts: An "Area Flow-Guided" prompt aggregates regional traffic flows. A Road Condition Passing Mechanism estimates traffic at missing points by interpolating conditions from observed neighbors.
  4. Interval-Aware Trajectory Embedder: This component unifies trajectories of varying intervals into a standard high-frequency sequence by inserting [m] placeholders. It employs Learnable Fourier Features (LFF) to map continuous coordinates into high-dimensional embeddings.
  5. PLM-based Encoder: The core is a BERT-small model adapted via LoRA (Low-Rank Adaptation). This allows the model to be fine-tuned efficiently on limited dense data without overfitting.

Experimental Evaluation

The model was evaluated on two large-scale datasets: Chengdu (118,354 trajectories) and Porto (322,079 trajectories). Both were standardized to a 15-second ground truth interval and downsampled to 1, 2, and 4-minute intervals for testing.

Performance comparison of PLMTrajRec against baseline methods across different sampling intervals.

As shown in the results, PLMTrajRec consistently outperforms 12 baseline methods, including state-of-the-art models like RNTrajRec and MM-STGED. Key findings include:

  • Accuracy and Precision: PLMTrajRec achieved significant improvements in accuracy, particularly in the most challenging 4-minute sampling scenario.
  • Error Reduction: In the Chengdu dataset, the model reduced the Mean Absolute Error (MAE) by an average of 351.8 meters compared to baselines.
  • Scalability: The model demonstrated remarkable data efficiency; when trained on only 20% of the data, it still outperformed most baselines trained on the full dataset.

Conclusion

PLMTrajRec demonstrates that Pre-trained Language Models are powerful spatial reasoners capable of capturing the "grammar" of movement. By integrating natural language prompts with spatiotemporal embeddings, the framework achieves superior accuracy and generalization across varying sampling intervals. Future work will explore the integration of larger foundation models and dynamic road network updates to further enhance recovery performance in evolving urban environments.

Summary updated

Abstract

Spatiotemporal trajectory data is crucial for various applications. However, issues such as device malfunctions and network instability often cause sparse trajectories, leading to lost detailed movement information. Recovering the missing points in sparse trajectories to restore the detailed information is thus essential. Despite recent progress, several challenges remain. First, the lack of large-scale dense trajectory data makes it difficult to train a trajectory recovery model from scratch. Second, the varying spatiotemporal correlations in sparse trajectories make it hard to generalize recovery across different sampling intervals. Third, the lack of location information complicates the extraction of road conditions for missing points. To address these challenges, we propose a novel trajectory recovery model called PLMTrajRec. It leverages the scalability of a pre-trained language model (PLM) and can be fine-tuned with only a limited set of dense trajectories. To handle different sampling intervals in sparse trajectories, we first convert each trajectory's sampling interval and movement features into natural language representations, allowing the PLM to recognize its interval. We then introduce a trajectory encoder to unify trajectories of varying intervals into a single interval and capture their spatiotemporal relationships. To obtain road conditions for missing points, we propose an area flow-guided implicit trajectory prompt, which models road conditions by collecting traffic flows in each region. We also introduce a road condition passing mechanism that uses observed points' road conditions to infer those of the missing points. Experiments on two public trajectory datasets with three sampling intervals each demonstrate the effectiveness, scalability, and generalization ability of PLMTrajRec.
BibTeX
@inproceedings{wei2025plmtrajrec,
  author = {Tonglong Wei and Yan Lin and Youfang Lin and Shengnan Guo and Jilin Hu and Cong Gao and Huaiyu Wan and Wan, Huaiyu},
  title = {{PLMTrajRec: A Scalable and Generalizable Trajectory Recovery Method with Pre-trained Language Models}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=SJPq1xBPHX},
}
Figure