TL;DR
This paper benchmarks various statistical and machine learning models for predicting MBTA subway ridership and daily delays. Surprisingly, calendar features like day of the week and season proved far more impactful than weather data, which sometimes even degraded performance, suggesting overfitting. A novel application of a self-exciting Hawkes point process model was also explored for delay prediction.
What Problem Does This Paper Address?
Public transportation systems, like Boston's MBTA, are crucial for urban mobility but often suffer from unpredictable fluctuations in ridership and frequent delays. These issues negatively impact efficiency and passenger satisfaction. To address this, the paper aims to understand and predict two key aspects of MBTA operations: daily subway ridership (using gated station entries as a proxy) and the total number of daily delays across the system. The challenge lies in identifying which factors—calendar-based (day of week, season) or environmental (weather)—are most influential in these predictions and in comparing the performance of various modeling approaches.
Methods
The study evaluates a suite of 10 statistical and machine learning models for predicting next-day subway usage and delays. These include established methods like Random Forest, Gradient Boosting, Linear Regression, Ridge, Lasso, Support Vector Machines, Multilayer Perceptrons (MLP), and K-Nearest Neighbors. A simple moving average baseline is also included. For delay prediction, an 11th model, a self-exciting Hawkes point process, is introduced, offering a novel application of this framework to model the temporal clustering of delay events.
A key aspect of the methodology is a systematic exploration of feature importance. Models are trained and evaluated using various combinations of feature groups: day-of-week indicators, seasonal indicators, and four weather variables (pressure, wind speed, precipitation, temperature). This allows for an assessment of the incremental value of each data type. Model performance is primarily measured using Root Mean Squared Error (RMSE). Feature importance is further analyzed using SHAP (SHapley Additive exPlanations) values.
Datasets
The study utilizes two primary datasets:
- MBTA Service Alerts Dataset: This dataset provides daily delay data for the MBTA system, spanning from January 2019 to July 2023.
- MBTA Gated Station Entries Dataset: This dataset contains daily ridership data (proxy for subway usage) from January 2014 to June 2025.
Weather data, including pressure, wind speed, average temperature, and precipitation, was sourced from Meteostat, specifically for Boston Logan International Airport, serving as a proxy for Boston weather.
Key Insights and Ideas
- Calendar Beats Weather: The most striking finding is the overwhelming importance of calendar-based features (day of week, season) over weather data for predicting both ridership and delays. This suggests that human schedules and seasonal patterns are stronger drivers of transit demand and disruption than immediate weather conditions.
- Weather Can Hurt: Counterintuitively, including weather data sometimes worsened model performance, indicating a tendency for models to overfit to noisy weather signals rather than capturing genuine predictive power.
- Consistent Top Performers: A core group of models—Random Forest, Gradient Boosting, and Multilayer Perceptrons—consistently performed best across both delay prediction and ridership forecasting tasks, highlighting their robustness for time-series prediction.
- Hawkes for Clustering: While the Hawkes point process model didn't outperform simpler methods for daily aggregate delay prediction, it offers valuable insights into the self-exciting nature of delays, where one delay can trigger subsequent ones, making it potentially useful for real-time operational adjustments.
- Weekly Trends are Key: SHAP analysis confirms that day-of-week features are highly influential, suggesting that MBTA transit dynamics exhibit strong repeating weekly patterns that models can effectively leverage.
Key Results
- Model Performance: Random Forest, Gradient Boosting, and Multilayer Perceptrons were identified as the top-performing models for both delay prediction and gated station entry prediction, achieving significantly better accuracy when provided with additional data.
- Feature Importance: Day-of-week features consistently showed the greatest positive impact on predictive accuracy for both tasks. Conversely, weather data often had a negligible or even negative impact. For delay prediction, "total delays 1 day before" was the most important factor, while for ridership, "gated station entries 1 day before" was paramount.
- Hawkes Model Performance: The Hawkes model achieved an RMSE of 137.43 counts/day for daily delay aggregation and 0.670 hours for next-event delay prediction. While it trailed a simple moving average for daily counts, its interpretable parameters quantify the self-exciting nature of delays, with a branching ratio of 0.90 and a half-life of 0.33 hours, indicating that delays significantly increase the chance of subsequent delays within a short timeframe.
Limitations and Future Work
Limitations: * The study acknowledges that simpler time series models like ARIMA were not included due to preliminary observations suggesting poor performance on daily aggregates, likely due to the law of large numbers effect with a high number of daily events. * The paper notes that the current analysis aggregates most inputs at the daily level, which might miss finer-grained temporal dynamics.
Future Work: * Increased Resolution: Moving to hourly or station-level data resolution could uncover more localized effects and improve model accuracy. * Operational Variables: Incorporating operational data such as train assignments, crew shifts, maintenance schedules, and major events could explain more residual variation and enhance forecasts. * Spatial Network Structure: Embedding spatial network topology and transfer hub information into models and the Hawkes kernel could better capture how disruptions propagate through the system. * Real-time Applications: The Hawkes model's ability to quantify short-term clustering of disruptions suggests its potential for real-time dispatch and crowding reduction.
Summary updated