Abstract
Learning universal representations for check-in sequences is challenging due to temporal uncertainty, spatial diversity, and the friction between fusing disparate data types. We present STCCR, a novel pre-training framework that aligns high-level "spatial topics" with "temporal intentions." By leveraging contrastive clustering and an angular margin loss, the model effectively filters out noise and captures the macroscopic patterns of human mobility, outperforming state-of-the-art models in location, time, and user-link prediction tasks.
Introduction
Human mobility data, specifically check-in sequences from services like Yelp or Gowalla, is notoriously messy. Current models struggle to learn generalized representations due to three primary hurdles:
- Temporal Uncertainty: User arrival times are often fuzzy (e.g., arriving for dinner between 16:45 and 17:15). Precise timestamp embedding fails to capture this inherent noise.
- Spatial Diversity: Different users visiting different Points of Interest (POIs) may share the same high-level "topic" (e.g., "working in an office"). Tokenizing every POI uniquely misses these semantic similarities.
- Fusion Friction: Spatial and temporal data have distinct properties. Early fusion can introduce noise, while late fusion may lack semantic alignment.
Methodology: The STCCR Framework
The Spatial-Temporal Cross-view Contrastive Representation (STCCR) framework addresses these issues through a decoupled pre-training strategy consisting of three main modules:
Spatial Topic Module (STM)
Instead of raw POI IDs, the STM uses contrastive clustering to group sequences into "spatial topics" (e.g., leisure vs. work areas). This allows the model to recognize shared mobility behaviors across different sets of POIs.
Temporal Intention Module (TIM)
To handle temporal noise, the authors introduce a Time Angular Margin (TAM) loss. By adding an additive angular margin during contrastive learning, the model creates robust decision boundaries against small temporal perturbations. It also incorporates a Social Aware Block using Graph Attention Networks (GATs) to account for social circle influences.
ST Cross-View Contrastive Module
This module aligns spatial and temporal views in a unified semantic space. By treating one view as a reference for the other, the model learns deep correlations between where people go and when they go there.
Experimental Evaluation
The model was evaluated on three real-world check-in datasets: Gowalla-NYC, Gowalla-TKY, and WeePlace. All datasets were filtered to ensure quality, removing users with fewer than 10 records.
Key Results
STCCR was tested against several baselines (e.g., DeepMove, PLSPL) across three tasks:
- Next Location Prediction (LP): STCCR improved Acc@5 by an average of 3.9%.
- Trajectory User Link (TUL): The model saw a 4.3% improvement in Acc@5.
- Next Time Prediction (TP): A standout result with a 7.9% reduction in Mean Absolute Error (MAE), validating the TAM loss.
Discussion and Insights
- Clustering as Regularization: Forcing sequences into clusters prevents the model from memorizing specific POI sequences, boosting generalization.
- Angular Margins for Robustness: The TAM loss provides a "buffer zone," ensuring that minor delays do not drastically alter the representation of a user's intention.
- Cross-View Synergy: Spatial and temporal signals are highly complementary. Aligning them at a macroscopic semantic level provides stronger self-supervision than interleaved sequence modeling.
Conclusion
While STCCR demonstrates significant improvements, it remains sensitive to hyperparameter tuning (e.g., number of clusters) and requires explicit social network data for its Social Aware Block. Future work may explore dynamic clustering or latent social relationship inference to further enhance the model's flexibility.
Summary updated