BERT4Traj: Transformer-Based Trajectory Reconstruction for Sparse Mobility Data

Hao Yang, Xiaobai Angela Yao, Christopher C. Whalen, Gengchen MaiGIScience 2025 · DOI

Abstract

Human mobility data, such as Call Detail Records (CDRs) and intermittent GPS pings, are often sparse and incomplete, hindering their utility for urban planning and public health. This paper introduces BERT4Traj, a transformer-based architecture designed to reconstruct continuous trajectories from such "holey" data. By leveraging bidirectional self-attention and incorporating contextual background tokens—including demographics and temporal information—BERT4Traj significantly outperforms traditional recurrent neural networks and Markov models. Our evaluation on real-world datasets from Kampala, Uganda, demonstrates a substantial improvement in reconstruction accuracy, reaching 87.1% for CDR data.

Introduction

Understanding urban mobility is critical for infrastructure development and disease modeling. However, the primary sources of this data are often fragmented. Call Detail Records (CDRs) only capture a user's location during active communication, leaving large temporal gaps. Similarly, GPS data can be sparse due to signal interference or power-saving protocols.

Traditional reconstruction methods, such as Markov Chains or linear interpolation, often fail to capture the complex, non-linear dependencies inherent in human behavior. This paper addresses the challenge of bridging these gaps to create a high-resolution, continuous view of individual mobility patterns.

Methodology

The proposed model, BERT4Traj, adapts the Bidirectional Encoder Representations from Transformers (BERT) architecture to the domain of spatial trajectories. It treats a sequence of visited locations as a "sentence," where each location serves as a "word."

Model Architecture

The core of BERT4Traj is a Transformer Encoder that utilizes bidirectional attention, allowing the model to consider both past and future recorded visits when predicting a missing location.

Architecture of the BERT4Traj model, highlighting the integration of background tokens and trajectory embeddings.

Key technical components include:

  • Multi-Faceted Embeddings: The model uses Space2Vec for spatial encoding of coordinates and sinusoidal positional encodings for temporal periodicity.
  • Contextual Background Tokens: Uniquely, the model incorporates non-trajectory data such as age, gender, and "anchor points" (home and work locations) as input tokens.
  • Masked Modeling: During training, the model learns to predict randomly masked locations within a trajectory, forcing it to learn the underlying patterns of movement.

Experimental Results

The model was evaluated using two datasets from Kampala, Uganda: a sparse CDR dataset (248 participants) and a higher-resolution GPS dataset (586 participants).

BERT4Traj consistently outperformed all baselines, including Markov Chains, K-Nearest Neighbors (KNN), and Long Short-Term Memory (LSTM) networks.

Performance comparison of BERT4Traj against traditional baseline models across CDR and GPS datasets.

As shown in the results, BERT4Traj achieved an accuracy of 87.1% on the CDR dataset, a significant improvement over the 74.5% achieved by the best-performing baseline (LSTM). For GPS data, the model reached 71.4% accuracy. The lower absolute accuracy in GPS is attributed to the increased complexity of predicting specific 100m grid cells compared to broader cell tower coverage areas. Furthermore, the correct location was found within the model's top 5 predictions 91.2% of the time for CDR data.

Discussion and Key Insights

The success of BERT4Traj highlights several critical aspects of trajectory reconstruction:

  1. Bidirectional Context: Knowing a user's destination in the evening provides vital clues about their midday location. The Transformer's ability to "look ahead" is a significant advantage over unidirectional RNNs.
  2. Contextual Anchors: Human movement is not random; it is tethered to social roles and fixed locations. Incorporating demographics and home/work anchor points provides the "why" behind the "where," leading to higher predictive power.
  3. Temporal Rhythms: Human mobility is deeply rhythmic. Ablation studies confirmed that temporal information (date and time) is the strongest signal for accurate reconstruction.

Conclusion and Future Work

BERT4Traj represents a significant advancement in reconstructing human mobility from sparse data. While the results are promising, future work should address the model's generalizability to different urban environments and the privacy implications of high-accuracy trajectory reconstruction. Integrating multi-source data—such as combining CDR with occasional GPS pings—remains a promising avenue for further enhancing reconstruction robustness.

Summary updated

BibTeX
@inproceedings{yang2025bert,
  author = {Hao Yang and Xiaobai Angela Yao and Christopher C. Whalen and Gengchen Mai},
  title = {{BERT4Traj: Transformer-Based Trajectory Reconstruction for Sparse Mobility Data}},
  booktitle = {GIScience},
  year = {2025},
  doi = {10.4230/lipics.giscience.2025.8},
}
Figure