LLM Agents for Smart City Management: Enhancing Decision Support Through Multi-Agent AI Systems

Anna V. Kalyuzhnaya, Sergey Mityagin, Ekaterina Lutsenko, Andrey Getmanov, Yaroslav Aksenkin, Kamil Fatkhiev, Kirill Fedorin, Nikolay O. Nikitin, Natalia Chichkova, Vladimir Vorona, Alexander V. BoukhanovskySmart 2025 · DOI

TL;DR

This paper introduces a multi-agent LLM system designed to enhance decision-making in smart city management. By integrating LLMs with existing urban data and services, the system achieves significantly higher accuracy and efficiency in processing complex urban planning queries compared to standalone LLMs, paving the way for more informed urban governance.

What Problem Does This Paper Address?

Modern cities generate vast amounts of complex, heterogeneous data from various sources like urban planning documents, geospatial information, and real-time service APIs. Extracting actionable insights from this data for strategic decision-making is a significant challenge. While Large Language Models (LLMs) show promise in processing unstructured text, they often struggle with domain-specific knowledge, real-time data, and can "hallucinate" information. This paper addresses the gap in effectively leveraging LLMs for complex urban management tasks by proposing a multi-agent system that combines LLM capabilities with structured access to urban data and services, aiming to improve both the accuracy and efficiency of decision support.

Methods

The core of the proposed solution is a multi-agent system architecture. Individual LLM agents are designed with mechanisms for instruction processing, action execution, and reflection. These agents interact through two primary means: "tool usage" (where an agent calls external functions or APIs) and "agent-to-agent communication" (peer-to-peer interaction).

The system is structured into three layers:

  1. Interface Layer: A chatbot interface for user interaction.
  2. Orchestration Layer: An "agent-conductor" that coordinates the system's operations.
  3. Processing Layer: Specialized secondary agents for:
  4. Tool Calling: Interacting with city services via APIs.
  5. RAG Operations: Performing Retrieval-Augmented Generation using a vector database of municipal documents.
  6. Answer Summarization: Synthesizing and formatting final responses.
  7. Validation: Ensuring the integrity of answers.

The system utilizes Retrieval-Augmented Generation (RAG) to ground LLM responses in factual information from document databases and integrates with city service APIs for real-time data.

Datasets

The paper uses a validation dataset of 150 question-answer pairs, manually prepared by urban studies experts and augmented using GPT-40. This dataset is available in an open repository. The system also leverages existing data sources such as OpenStreetMap, the Territorial Development Fund's open data, and national statistics service data for St. Petersburg.

Key Insights and Ideas

  • Multi-agent systems are key: Standalone LLMs struggle with the complexity and data requirements of smart city management. A multi-agent approach, where specialized agents handle different tasks (data retrieval, RAG, tool calling), is crucial for success.
  • Context is king (especially for numbers): LLMs without access to relevant, factual context perform poorly, especially when numerical precision is required. Providing context significantly boosts accuracy.
  • RAG is a powerful tool: Retrieval-Augmented Generation, especially when combined with local knowledge and regulations, dramatically improves response quality and reduces the tendency for LLMs to "hallucinate."
  • Middle-sized LLMs can be sufficient: The study suggests that middle-sized LLM models can achieve high accuracy, potentially outperforming larger models when integrated into a well-designed multi-agent system.
  • Hybrid approach for optimal results: While LLM multi-agent systems offer remarkable speed and efficiency for data analysis, human expertise remains vital for strategic, long-term planning and decision-making.

Key Results

  • Pipeline Selection Accuracy: The multi-agent system achieved pipeline selection accuracy of 94-99% across different LLM models.
  • Response Accuracy Improvement:
    • RAG integration improved response accuracy by 17% for strategic development queries and 55% for service accessibility questions.
  • The combined use of document databases and service APIs resulted in the highest G-Eval scores of 0.68–0.74, significantly outperforming standalone LLM responses (0.30–0.38).
  • Efficiency: The system can complete tasks in hours that would typically take human teams days or weeks.
  • Relevance: The system maintained high relevance in its answers (scores of 0.91–0.95).

Limitations and Future Work

  • Computational Scalability: Deploying LLMs requires substantial computational resources, posing a trade-off between cloud infrastructure (with security risks) and local deployment (with high hardware investment).
  • Real-time Data Integration: Ensuring consistent, secure, and standardized data flow from diverse urban information systems remains a challenge.
  • Agent System Stability: The reliability of the complex orchestration between agents, especially in production environments, is a concern, requiring careful prompt engineering and regular updates.
  • Future Work:
    • Enhancing agent coordination for complex urban planning workflows.
  • Improving validation mechanisms for factual accuracy.
  • Optimizing computational efficiency for resource-constrained municipalities.
  • Developing specialized agents for emergency response and crisis management.
  • Integrating real-time urban sensing networks.
  • Improving temporal modeling capabilities for urban evolution tracking.
  • Phased implementation for cities with varying digital maturity.

Summary updated

Abstract

This study investigates the implementation of LLM agents in smart city management, leveraging both the inherent language processing abilities of LLMs and the distributed problem solving capabilities of multi-agent systems for the improvement of urban decision making processes. A multi-agent system architecture combines LLMs with existing urban information systems to process complex queries and generate contextually relevant responses for urban planning and management. The research is focused on three main hypotheses testing: (1) LLM agents’ capability for effective routing and processing diverse urban queries, (2) the effectiveness of Retrieval-Augmented Generation (RAG) technology in improving response accuracy when working with local knowledge and regulations, and (3) the impact of integrating LLM agents with existing urban information systems. Our experimental results, based on a comprehensive validation dataset of 150 question–answer pairs, demonstrate significant improvements in decision support capabilities. The multi-agent system achieved pipeline selection accuracy of 94–99% across different models, while the integration of RAG technology improved response accuracy by 17% for strategic development queries and 55% for service accessibility questions. The combined use of document databases and service APIs resulted in the highest performance metrics (G-Eval scores of 0.68–0.74) compared to standalone LLM responses (0.30–0.38). Using St. Petersburg’s Digital Urban Platform as a testbed, we demonstrate the practical applicability of this approach to create integrated city management systems with support complex urban decision making processes. This research contributes to the growing field of AI-enhanced urban management by providing empirical evidence of LLM agents’ effectiveness in processing heterogeneous urban data and supporting strategic planning decisions. Our findings suggest that LLM-based multi-agent systems can significantly enhance the efficiency and accuracy of urban decision making while maintaining high relevance in responses.
BibTeX
@inproceedings{kalyuzhnaya2025llm,
  author = {Anna V. Kalyuzhnaya and Sergey Mityagin and Ekaterina Lutsenko and Andrey Getmanov and Yaroslav Aksenkin and Kamil Fatkhiev and Kirill Fedorin and Nikolay O. Nikitin and Natalia Chichkova and Vladimir Vorona and Alexander V. Boukhanovsky},
  title = {{LLM Agents for Smart City Management: Enhancing Decision Support Through Multi-Agent AI Systems}},
  booktitle = {Smart},
  year = {2025},
  doi = {10.3390/smartcities8010019},
}
Figure