TL;DR
TrajPrism is a new large-scale, multi-task benchmark designed to bridge the gap between spatial urban mobility and natural language understanding. By pairing 300K real-world GPS trajectories across three major cities with 2.1 million language instances, it challenges models to perform bidirectional tasks: generating trajectories from instructions, retrieving trajectories via semantic queries, and captioning routes. The paper reveals that traditional, geometry-only mobility models fail significantly when language is introduced into the interface, establishing language-trajectory alignment as an open challenge.
What Problem Does This Paper Address?
Urban mobility analysis has traditionally been split into two isolated domains. On one side, trajectory modeling is highly geometry-centric, focusing on coordinates, distances, and shapes while ignoring human intent and context. On the other side, recent language-centric mobility benchmarks evaluate large language models (LLMs) on high-level route planning or tool use, but they lack fine-grained, verifiable alignment with actual physical routes.
What is missing is a unified framework that evaluates both modalities together on real-world data. To address this, the authors introduce TrajPrism, a benchmark that grounds unconstrained natural language directly into real urban trajectories, forcing models to understand travel intents, constraints, and preferences.
Methods
The authors developed a reproducible data generation pipeline called Reverse Intent Reconstruction (RIR) to synthesize high-quality language annotations from raw GPS traces:
- Phase-based Semantic Route Abstraction: Raw trajectories are compressed into semantic "phases" using Uber’s H3 hexagonal hierarchical spatial index. This run-length encoding aggregates headings, durations, street names, and nearby Points of Interest (POIs).
- Intent Sampling: Travel intent is sampled from a four-dimensional taxonomy: Destination, Waypoint, Route Preference, and Temporal/Pace.
- Multi-task Generation: Using a mix of Gemini 3.1 Flash, Claude 4.6 Sonnet, and GPT-OSS 120B, the pipeline generates three instruction variants (Literal, Concise, Chatty), three retrieval queries, and one factual caption per trajectory.
- Quality Control: A five-stage deterministic filter is paired with a cascaded LLM-and-human judging process to eliminate hallucinations and ensure spatial accuracy.
To calibrate the benchmark, the authors also introduced three baseline proof-of-concept models:
- TrajAnchor (Generation): A RAG-based pipeline that retrieves a similar trajectory as a spatial anchor, extracts constraints using an LLM, and routes the path via a soft-weighted chain Dijkstra search.
- TrajFuse (Retrieval): A dual-encoder contrastive framework that fuses geometric trajectory embeddings with H3 semantic text descriptions, aligning them with text queries using an InfoNCE loss.
- TrajRap (Captioning): A retrieval-augmented few-shot prompting pipeline that generates factual captions from structural features and semantic context.
Datasets
The newly introduced TrajPrism benchmark consists of:
- 300K real-world trajectories sampled across three distinct cities: Porto, San Francisco, and Beijing.
- 2.1M total task instances (7 annotations per trajectory).
Unlike existing benchmarks like Semantic Routing (which uses synthetic queries) or MobilityBench (which focuses on API tool use for agent planning), TrajPrism is the only dataset built from real GPS trajectories that supports bidirectional language-to-trajectory and trajectory-to-language tasks.
Key Insights and Ideas
- Geometry-Only Models Fall Short: Purely geometric trajectory encoders (like UniTraj or BLUE) completely fail at language-grounded tasks because they cannot capture abstract human preferences or temporal constraints.
- Task-Specific Bottlenecks:
- Generation is primarily bottlenecked by destination selection. When an oracle provides the correct destination, trajectory fidelity (Jaccard similarity) nearly doubles.
- Retrieval struggles heavily with implicit temporal or preference-based queries (e.g., "avoid high traffic"), whereas explicit spatial queries are handled much better.
- Captioning models are highly fluent but suffer from poor spatial grounding; they struggle to correctly recall specific POIs without direct access to map-level semantic context.
- Map-Level Semantics as an Enabler: Integrating structured spatial knowledge (like H3 cell area descriptions) consistently boosts performance across all three tasks, serving as the necessary bridge between raw coordinates and natural language.
Key Results
- Task 1 (Trajectory Generation): Even the best-performing TrajAnchor variant achieved only a 17–20% Destination Hit Rate and a low Jaccard similarity (~0.25–0.30) across cities, underscoring the difficulty of the task.
- Task 2 (Semantic Retrieval): TrajFuse outperformed all text-only and geometry-only baselines. However, its Hard Recall@1 remained below 0.10 in Porto and below 0.23 in other cities, leaving massive headroom for improvement.
- Task 3 (Trajectory Captioning): While language fluency scores were high (BERTScore F1 > 0.83), structural-only baselines achieved a dismal POI Recall of just 0.015 in San Francisco. Adding semantic context (TrajRap) dramatically improved POI Recall to 0.341 in Porto.
Limitations and Future Work
The authors note several limitations that pave the way for future research:
- Language and City Diversity: The benchmark currently features English-only annotations and covers only three cities.
- Synthesis Bias: The travel intents are LLM-synthesized, which might introduce distributional biases compared to organic, human-generated user queries.
- Scope Constraints: TrajPrism is restricted to map-matched vehicle GPS traces. It does not yet address indoor navigation, pedestrian routing, or multimodal transportation networks.
Summary updated