Abstract
Urban commuting data has long been a vital source of understanding population mobility behaviour and has been widely adopted for various applications such as transport infrastructure planning and urban anomaly detection. While individual-specific transaction records (such as smart card (tap-in, tap-out) data or taxi trip records) hold a wealth of information, these are often private data available only to the service provider (e.g., taxicab operator). In this work, we explore the utility in harnessing publicly available, albeit noisy, transportation datasets, such as noisy "Estimated Time of Arrival" (ETA) records (commonly available to commuters through transit Apps or electronic signages). We first propose a framework to extract accurate individual bus trajectories from such ETA records, and present results from both a primary city (Singapore) and a secondary city (London) to validate the techniques. Finally, we quantify the upper bound on the spatiotemporal resolution, of the reconstructed trajectory outputs, achieved by our proposed technique.
Introduction
Understanding urban mobility relies heavily on tracking the movement of people and vehicles. Traditionally, high-resolution tracking requires access to privileged datasets like smart card transaction logs or proprietary GPS traces from transit fleets. However, these datasets are often restricted due to privacy concerns or released with significant latency, limiting their utility for real-time applications.
In contrast, public transit APIs provide real-time Estimated Time of Arrival (ETA) data. While accessible, this data is aggregated, noisy, and lacks unique vehicle identifiers. An API might report multiple upcoming buses for a stop without specifying which physical vehicle corresponds to which record. This paper addresses the challenge of reverse-engineering these anonymous snapshots to infer continuous, individual bus trajectories.
Methodology
The researchers developed a three-step pipeline to transform raw, anonymous ETA snapshots into unique bus trajectories.
1. De-duplication
Because ETA APIs report data for multiple upcoming stops, a single physical bus often appears multiple times in a single snapshot (e.g., as the "next bus" for one stop and the "second bus" for another). The algorithm employs distance and ETA-based heuristics to group these redundant observations and assign them temporary local identifiers.
2. Stitching
The next phase involves matching these local IDs across consecutive time snapshots. The algorithm calculates the distance traveled between snapshots and applies speed-based sanity checks—for instance, discarding matches that imply unrealistic speeds in dense urban traffic—to maintain a continuous trajectory for each vehicle.
3. Interpolation
De-duplication naturally removes redundant data points, which can lead to gaps in the trajectory at certain stops. To address this, the framework uses historical average velocities for specific routes and inter-stop distances to interpolate the exact times a bus passed missing stops. This step is a significant performance driver, reducing arrival estimation errors by up to 30%.
Datasets and Validation
The framework was validated using two primary datasets:
- Singapore (DataMall API): The primary testbed, featuring 60-second polling across over 4,000 stops. It provides ETA and crowdedness data for the next three buses.
- London (TfL Unified API): A secondary dataset with 30-second polling. This dataset includes ground-truth vehicle IDs, serving as a robust validation set for the tracking algorithm.
Ground truth was further established through 28 hours of manual observation (3,173 bus crossings) in Singapore and historical smart-fare transaction data to verify bus frequencies.
Key Results
The proposed method demonstrates that public ETA data is sufficient for high-fidelity tracking:
- Tracking Accuracy: The algorithm achieves over 80% accuracy in matching official transit schedules and over 80% precision/recall in detecting stop-crossing times within a one-minute margin.
- Spatiotemporal Resolution: The system achieves a spatial tracking resolution of approximately 200 to 600 meters with 80% confidence.
- Predictive Performance: Real-time inferred trajectories significantly outperform historical averages for downstream arrival prediction. In testing, 75% of predictions were within a 45-second error margin, compared to only 55% when using historical data. Gains were most pronounced when polling intervals were shorter (e.g., 30 seconds vs. 2 minutes).
Limitations and Future Work
Challenges remain in scenarios of "bus bunching," where closely following vehicles may be incorrectly clustered. Additionally, skipped stops (occurring in roughly 25% of weekday trips) and mid-route activations can introduce distance calculation errors. Future research could integrate inferred trajectories with occupancy data to model urban crowd flows and transit bottlenecks in real-time.
Summary updated