TraveLLM: Could You Plan My Public Transit Alternatives in Face of a Network Disruption?

Bowen Fang, Zifan Yang, Xuan Di2025 · DOI

TL;DR

This paper introduces TraveLLM, a novel system that uses Large Language Models (LLMs) to plan public transit routes during disruptions, incorporating real-time events and complex user preferences. By leveraging LLMs' reasoning and multimodal processing capabilities, TraveLLM aims to provide more flexible and personalized navigation solutions than traditional methods.

What Problem Does This Paper Address?

Current navigation systems often falter when faced with urban disruptions like severe weather, emergencies, or dynamic service changes. They struggle to integrate real-time event information with complex user constraints, such as avoiding specific areas due to safety concerns or personal preferences. Traditional route planning algorithms, like Dijkstra's, are optimized for stable networks and lack the flexibility to handle these dynamic, personalized, and often colloquially expressed needs. Furthermore, multimodal planning, which combines different transportation options (e.g., subway, bus, bike-share), becomes significantly more challenging during disruptions. This paper addresses the gap in creating intelligent navigation systems that can adapt to these complex, real-world scenarios.

Methods

TraveLLM employs a two-stage LLM architecture:

  1. LLM Planner (Lplanner): This stage takes user queries (including origin, destination, disruption information, and preferences) and a knowledge base (containing transportation network data and general constraints) as input. It uses LLMs like GPT-4, Claude 3, and Gemini to reason over this information and generate a detailed, natural language travel plan. The prompt engineering guides the LLM to prioritize safety and efficiency.
  2. LLM Summary (Lsummary): This stage takes the detailed plan from the Lplanner and structures it into a predefined, concise format. This separation allows the planner to focus on generating a high-quality plan without being constrained by strict output formatting, while the summarizer ensures the output is reliable and structured. GPT-4 is used for this stage due to its strong instruction-following capabilities.

The system's effectiveness is evaluated using metrics like connectivity, constraint avoidance, normalized travel time, and number of transfers.

Datasets

The paper does not explicitly mention training datasets. Instead, it focuses on evaluating LLMs using a custom-designed benchmark suite of test cases. These scenarios are based on realistic disruption situations, including:

  • North-South subway disruptions (e.g., extreme weather).
  • Cross-river and cross-town disruptions (e.g., emergencies).
  • Scenarios involving visual avoidance zones (requiring interpretation of map images).
  • Multimodal planning scenarios integrating subway, bus, and bike-share data.
  • A test case using the Washington D.C. Metro system to assess generalizability.

This benchmark suite is designed to probe specific LLM capabilities like reasoning under disruptions, constraint handling, and multimodal information processing.

Key Insights and Ideas

  • LLMs as Disruption-Aware Planners: The core idea is that LLMs, with their advanced reasoning and natural language understanding, are well-suited to tackle the complexity of disruption-aware navigation, a task that traditional algorithms struggle with.
  • Two-Stage Architecture for Robustness: Separating the planning and summarization tasks significantly improves the quality and reliability of the output. The planner can focus on generating a coherent plan, while the summarizer ensures it adheres to a structured format, preventing errors and extraneous text.
  • Value of Visual Context: Incorporating map images (even simple ones like avoidance zones) into the LLM's knowledge base dramatically improves planning performance across multiple metrics. This highlights the importance of visual-spatial reasoning in navigation.
  • LLM Variability: Different LLMs exhibit varying strengths. GPT-4 emerged as the most effective overall planner, but Claude 3 Opus showed particular promise in handling complex multimodal scenarios, suggesting specialized LLMs might be beneficial for specific tasks.
  • Qualitative Route Generation: Beyond just finding a route, LLMs can generate diverse, qualitative routes based on nuanced user preferences (e.g., "safest," "fastest," "scenic"), offering a more personalized and explainable navigation experience.

Key Results

  • GPT-4 as the Top Planner: In the LLM planner comparison, GPT-4 significantly outperformed Claude 3 Opus and Gemini Pro 1.0 in connectivity (0.78), avoidance (0.78), and normalized travel time (0.51).
  • Map Images Boost Performance: Providing map images to the GPT-4 planner improved connectivity by 0.11, avoidance by 0.34, reduced travel time by 0.08, and decreased transfers by 1.00.
  • Two-Stage Design Superiority: The two-agent (planner-summarizer) approach achieved substantially better planning metrics (Mconn, Mavoid, Mtime) and drastically reduced format violations (0.11 vs. 1.00) compared to a single-agent approach.
  • Multimodal Strength: Claude 3 Opus demonstrated a unique ability to handle complex multimodal scenarios involving bike-sharing, interpreting visual data for availability and planning logistics, which GPT-4 and Gemini struggled with.

Limitations and Future Work

  • Visual Constraint Complexity: Handling complex visual constraints, especially those requiring precise spatial reasoning, remains a challenge. No model perfectly satisfied both visual avoidance and path feasibility simultaneously in one scenario.
  • Multimodal Integration: While some LLMs show promise, effectively integrating less common transport modes and their associated data (like bike-share availability) is an area for improvement.
  • Generalizability: While the system was tested on a different transit system (Washington D.C. Metro), further work is needed to ensure robust performance across diverse urban environments.
  • Future Directions: The authors suggest future research focusing on improving visual reasoning, ensuring factual grounding of LLM outputs, and refining the integration of LLM-based planners into existing navigation applications. They also highlight the potential for LLMs to generate diverse, qualitative routes based on abstract user goals, which could be further explored.

Summary updated

Abstract

Existing navigation systems often fail during urban disruptions, struggling to incorporate real-time events and complex user constraints, such as avoiding specific areas. We address this gap with TraveLLM, a system using Large Language Models (LLMs) for disruption-aware public transit routing. We leverage LLMs' reasoning capabilities to directly process multimodal user queries combining natural language requests (origin, destination, preferences, disruption info) with map data (e.g., subway, bus, bike-share). To evaluate this approach, we design challenging test scenarios reflecting real-world disruptions like weather events, emergencies, and dynamic service availability. We benchmark the performance of state-of-the-art LLMs, including GPT-4, Claude 3, and Gemini, on generating accurate travel plans. Our experiments demonstrate that LLMs, notably GPT-4, can effectively generate viable and contextaware navigation plans under these demanding conditions. These findings suggest a promising direction for using LLMs to build more flexible and intelligent navigation systems capable of handling dynamic disruptions and diverse user needs.
BibTeX
@misc{fang2025travellm,
  author = {Bowen Fang and Zifan Yang and Xuan Di},
  title = {{TraveLLM: Could You Plan My Public Transit Alternatives in Face of a Network Disruption?}},
  year = {2025},
  doi = {10.1109/itsc60802.2025.11423075},
}
Figure