Flex-Route Transit for Smart Cities: A Reinforcement Learning Approach to Balance Ridership and Performance

Joseph Rodriguez, Haris N. Koutsopoulos, Jinhua ZhaoSmart 2025 · DOI

TL;DR

This paper introduces a reinforcement learning (RL) approach for flex-route transit systems to dynamically decide when to deviate from a fixed route to pick up passengers. By balancing the competing goals of increasing ridership and maintaining on-time performance, the RL method significantly outperforms a traditional heuristic approach, leading to better service accessibility and reliability in smart cities.

What Problem Does This Paper Address?

Traditional fixed-route public transit struggles to provide broad accessibility, especially in low-density urban areas. Flex-route transit aims to bridge this gap by combining the directness of fixed routes with the coverage of on-demand services, allowing vehicles to deviate for optional pickups. However, this flexibility creates a critical trade-off: deviating too much can lead to delays and negatively impact passengers at fixed stops, while deviating too little reduces ridership and accessibility. Existing methods often rely on static or reactive heuristics that can't effectively handle the dynamic nature of urban transit demand and travel times. This paper addresses the challenge of making real-time, optimal deviation decisions in flex-route transit to balance these competing objectives.

Methods

The core of this paper is a Multi-Agent Reinforcement Learning (MARL) approach, specifically using Deep Q-Networks (DQNs). The authors frame the problem as a Markov Decision Process (MDP) where individual vehicles act as agents.

  • State Space: The state for each vehicle includes its current location (fixed stop), the number of waiting requests at nearby flex stops, the current headway (time since the last vehicle), and its schedule deviation.
  • Action Space: Each vehicle can either deviate to a flex stop or continue on its fixed route. Action masking is used to prevent illogical actions (e.g., deviating when no requests are present).
  • Reward Function: A novel, weighted reward function is designed to balance two key objectives:
  • Request Acceptance: Penalizing skipped requests.
  • Schedule Status: Penalizing early and late arrivals, with a higher penalty for lateness due to its impact on transfers. The wbal parameter allows transportation authorities to tune the relative importance of these objectives based on their service priorities.

  • Training: The MARL agents are trained in a simulated environment that models dynamic demand and travel time variations. A shared policy is used to govern all agents, leveraging pooled experiences for faster convergence.

Datasets

The paper does not use pre-existing real-world datasets for training. Instead, it introduces a simulated case study based on a proposed flex-route service in Boston. This simulation incorporates:

  • A 2-mile route with 5 fixed stops and 2 flex stops in each direction.
  • Log-normal distributions for vehicle travel times between stops.
  • Poisson processes for passenger arrivals at stops.
  • Two distinct demand scenarios:
  • Peak Period: Characterized by commuter demand, prioritizing on-time arrivals.
  • Off-Peak Period: Characterized by leisure travelers, allowing for more flexibility and a higher tolerance for deviations.
  • A sensitivity analysis is conducted by varying travel time variability.

Key Insights and Ideas

  • The "Myopic vs. Forward-Looking" Advantage: The RL approach, by considering future rewards, makes more strategic deviation decisions than the heuristic, which is more reactive. This leads to a more balanced distribution of deviations along the route, not just focusing on the first available flex stop.
  • Dynamic Adaptation is Crucial: The RL agent's ability to adapt its deviation strategy based on real-time state information (requests, headway, schedule deviation) is key to its superior performance in dynamic urban environments.
  • Tunable Priorities for Smart Cities: The weighted reward function is a powerful tool, allowing city planners to explicitly define and adjust the balance between service accessibility (ridership) and reliability (on-time performance) based on specific urban contexts and goals.
  • Fixed-Stop Rider Benefits: Improvements in on-time performance and reduced wait times for fixed-stop riders demonstrate that flex-route optimization doesn't necessarily come at the expense of existing transit users.

Key Results

  • Improved On-Time Performance: MARL increased the on-time rate by 4% in the peak period and 6% in the off-peak period compared to the heuristic.
  • Higher Request Acceptance: MARL showed an increase in acceptance rate of 3.4% in the peak and a marginal decrease of 0.7% in the off-peak. The slight decrease in off-peak is attributed to the heuristic's more aggressive deviation behavior when demand is high.
  • Reduced Wait Times: MARL reduced the 90th percentile wait times for fixed-stop riders by 8% in the off-peak and 5% in the peak.
  • Robustness: The MARL approach demonstrated consistent performance improvements even under increased travel time variability, highlighting its adaptability.
  • Balanced Deviations: MARL made more balanced deviations along the route, with 38% at the second flex stop compared to 29% for the heuristic, indicating better anticipation of downstream opportunities.

Limitations and Future Work

  • Simplified Model: The current model does not consider vehicle capacity, drop-offs at flex stops, or dynamic schedule adjustments.
  • Action Space Extension: Future work could explore deferring requests to the next trip instead of outright rejection.
  • Coordination: Investigating more explicit multi-agent coordination techniques could further enhance performance.
  • Dynamic Weight Adjustment: Incorporating dynamic adjustment of reward weights within the RL policy could allow for adaptation to varying conditions within a single operational framework.
  • Continuous Simulation: Testing on longer, continuous-operation simulations would provide deeper insights into policy stability and long-term adaptability.

Summary updated

Abstract

A major challenge for modern transit systems relying on traditional fixed-route designs is providing broad accessibility to users. Flex-route transit can enhance accessibility in low-density areas, since it combines the directness of fixed-route transit with the coverage of on-demand mobility. Although deviating for optional pickups can increase ridership and transit accessibility, it also deteriorates the service performance for fixed-route riders. To balance this inherent trade-off, this paper proposes a reinforcement learning approach for deviation decisions. The proposed model is used in a case study of a proposed flex-route service in the city of Boston. The performance on competing objectives is evaluated for reward configurations that adapt to peak and off-peak scenarios. The analysis shows a significant improvement of our method compared to a heuristic derived from industry practice as a baseline. To evaluate robustness, we assess performance across scenarios with varying demand compositions (fixed and requested riders). The results show that the method achieves greater improvements than the baseline in scenarios with increased request ridership, i.e., where decision-making is more complex. Our approach improves service performance under dynamic demand conditions and varying priorities, offering a valuable tool for smart cities to operate flex-route services.
BibTeX
@inproceedings{rodriguez2025flex,
  author = {Joseph Rodriguez and Haris N. Koutsopoulos and Jinhua Zhao},
  title = {{Flex-Route Transit for Smart Cities: A Reinforcement Learning Approach to Balance Ridership and Performance}},
  booktitle = {Smart},
  year = {2025},
  doi = {10.3390/smartcities8050150},
}
Figure