Spatial-Temporal Cross-View Contrastive Pre-training for Check-in Sequence Representation Learning

Letian Gong, Huaiyu Wan, Shengnan Guo, Xiucheng Li, Yan Lin, Erwen Zheng, Tianyi Wang, Zeyu Zhou, Youfang Lin

Abstract

Learning universal representations for check-in sequences is challenging due to temporal uncertainty, spatial diversity, and the friction between fusing disparate data types. We present STCCR, a novel pre-training framework that aligns high-level "spatial topics" with "temporal intentions." By leveraging contrastive clustering and an angular margin loss, the model effectively filters out noise and captures the macroscopic patterns of human mobility, outperforming state-of-the-art models in location, time, and user-link prediction tasks.

Introduction

Human mobility data, specifically check-in sequences from services like Yelp or Gowalla, is notoriously messy. Current models struggle to learn generalized representations due to three primary hurdles:

  1. Temporal Uncertainty: User arrival times are often fuzzy (e.g., arriving for dinner between 16:45 and 17:15). Precise timestamp embedding fails to capture this inherent noise.
  2. Spatial Diversity: Different users visiting different Points of Interest (POIs) may share the same high-level "topic" (e.g., "working in an office"). Tokenizing every POI uniquely misses these semantic similarities.
  3. Fusion Friction: Spatial and temporal data have distinct properties. Early fusion can introduce noise, while late fusion may lack semantic alignment.

Methodology: The STCCR Framework

The Spatial-Temporal Cross-view Contrastive Representation (STCCR) framework addresses these issues through a decoupled pre-training strategy consisting of three main modules:

Spatial Topic Module (STM)

Instead of raw POI IDs, the STM uses contrastive clustering to group sequences into "spatial topics" (e.g., leisure vs. work areas). This allows the model to recognize shared mobility behaviors across different sets of POIs.

Temporal Intention Module (TIM)

To handle temporal noise, the authors introduce a Time Angular Margin (TAM) loss. By adding an additive angular margin during contrastive learning, the model creates robust decision boundaries against small temporal perturbations. It also incorporates a Social Aware Block using Graph Attention Networks (GATs) to account for social circle influences.

ST Cross-View Contrastive Module

This module aligns spatial and temporal views in a unified semantic space. By treating one view as a reference for the other, the model learns deep correlations between where people go and when they go there.

The overall STCCR architecture, showing the parallel spatial and temporal pipelines.

Experimental Evaluation

The model was evaluated on three real-world check-in datasets: Gowalla-NYC, Gowalla-TKY, and WeePlace. All datasets were filtered to ensure quality, removing users with fewer than 10 records.

Key Results

STCCR was tested against several baselines (e.g., DeepMove, PLSPL) across three tasks:

  • Next Location Prediction (LP): STCCR improved Acc@5 by an average of 3.9%.
  • Trajectory User Link (TUL): The model saw a 4.3% improvement in Acc@5.
  • Next Time Prediction (TP): A standout result with a 7.9% reduction in Mean Absolute Error (MAE), validating the TAM loss.

Discussion and Insights

  • Clustering as Regularization: Forcing sequences into clusters prevents the model from memorizing specific POI sequences, boosting generalization.
  • Angular Margins for Robustness: The TAM loss provides a "buffer zone," ensuring that minor delays do not drastically alter the representation of a user's intention.
  • Cross-View Synergy: Spatial and temporal signals are highly complementary. Aligning them at a macroscopic semantic level provides stronger self-supervision than interleaved sequence modeling.
The ST Cross-View module aligns spatial and temporal representations into a shared space.

Conclusion

While STCCR demonstrates significant improvements, it remains sensitive to hyperparameter tuning (e.g., number of clusters) and requires explicit social network data for its Social Aware Block. Future work may explore dynamic clustering or latent social relationship inference to further enhance the model's flexibility.

Summary updated

Abstract

The rapid growth of location-based services (LBS) has yielded massive amounts of data on human mobility. Effectively extracting meaningful representations for user-generated check-in sequences is pivotal for facilitating various downstream services. However, the user-generated check-in data are simultaneously influenced by the surrounding objective circumstances and the user's subjective intention. Specifically, the temporal uncertainty and spatial diversity exhibited in check-in data make it difficult to capture the macroscopic spatial-temporal patterns of users and to understand the semantics of user mobility activities. Furthermore, the distinct characteristics of the temporal and spatial information in check-in sequences call for an effective fusion method to incorporate these two types of information. In this paper, we propose a novel Spatial-Temporal Cross-view Contrastive Representation (STCCR) framework for check-in sequence representation learning. Specifically, STCCR addresses the above challenges by employing self-supervision from "spatial topic" and "temporal intention" views, facilitating effective fusion of spatial and temporal information at the semantic level. Besides, STCCR leverages contrastive clustering to uncover users' shared spatial topics from diverse mobility activities, while employing angular momentum contrast to mitigate the impact of temporal uncertainty and noise. We extensively evaluate STCCR on three real-world datasets and demonstrate its superior performance across three downstream tasks.
BibTeX
@article{gong2024spatial,
  author = {Letian Gong and Huaiyu Wan and Shengnan Guo and Xiucheng Li and Yan Lin and Erwen Zheng and Tianyi Wang and Zeyu Zhou and Youfang Lin},
  title = {{Spatial-Temporal Cross-View Contrastive Pre-training for Check-in Sequence Representation Learning}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2024},
  doi = {10.1109/tkde.2024.3434565},
}
Figure