RAISECity: A Multimodal Agent Framework for Reality-Aligned 3D World Generation at City-Scale

Shengyuan Wang, Zhiheng Zheng, Yu Shang, Lixuan He, Yangcheng Yu, Fan Hangyu, Jie Feng, Qingmin Liao, Yong Li

Abstract

City-scale 3D generation is of great importance for the development of embodied intelligence and world models. Existing methods, however, face significant challenges regarding quality, fidelity, and scalability in 3D world generation. Thus, we propose RAISECity, a Reality-Aligned Intelligent Synthesis Engine that creates detailed, City-scale 3D worlds. We introduce an agentic framework that leverages diverse multimodal foundation tools to acquire real-world knowledge, maintain robust intermediate representations, and construct complex 3D scenes. This agentic design, featuring dynamic data processing, iterative self-reflection and refinement, and the invocation of advanced multimodal tools, minimizes cumulative errors and enhances overall performance. Extensive quantitative experiments and qualitative analyses validate the superior performance of RAISECity in real-world alignment, shape precision, texture fidelity, and aesthetics level, achieving over a 90% win-rate against existing baselines for overall perceptual quality.

Introduction

Creating high-fidelity 3D cities is traditionally a labor-intensive process requiring extensive manual artistry. While AI has advanced 3D generation, existing methods struggle with a "trilemma" of scalability, fidelity, and reality alignment. Neural representations often fail to scale to thousands of buildings, while automated methods frequently produce oversimplified shapes or hallucinate layouts that do not match real-world geography. For applications like autonomous driving or urban planning, 3D worlds must precisely match GPS coordinates and architectural styles. RAISECity addresses these challenges by employing an agentic approach that orchestrates specialized tools for different stages of the construction process.

The RAISECity Framework

RAISECity functions as a multi-stage pipeline managed by an AI agent, moving away from end-to-end black-box generation toward a modular, tool-calling architecture.

Overview of the RAISECity agentic framework, illustrating the flow from perception and imagination to 3D generation and scene design.

The process is divided into several key modules:

  • Planning & Perception: The agent queries OpenStreetMap (OSM) for footprints and fetches panoramic street-view images, using models like Owl-ViT for object detection to identify the best visual references.
  • Imagination & Reflection: To overcome occlusions (like trees or cars) in real-world photos, the agent uses Gemini 2.5 Flash to "imagine" unobstructed building images. A Vision-Language Model (VLM) then acts as a critic in a "self-correction" loop, ensuring structural sanity and aesthetic quality before proceeding.
  • 3D Generation & Scene Design: Approved 2D references are converted into textured 3D meshes using the Hunyuan3D suite. Finally, these assets are imported into Blender, where OSM data guides their precise real-world placement alongside fine-grained elements like vegetation and street furniture.

Experimental Results

RAISECity significantly outperforms existing baselines such as CityCraft and UrbanWorld across multiple dimensions, as shown in qualitative comparisons.

Qualitative comparison of RAISECity against state-of-the-art baselines, demonstrating superior visual fidelity and alignment with real-world references.

Key performance highlights include:

  • Perceptual Quality: RAISECity achieved a 91% win rate in pairwise comparisons judged by humans and LLMs for overall quality.
  • Structural Fidelity: Metrics like Edge-IoU confirm that the generated building silhouettes align closely with real-world geometry.
  • Aesthetics: The framework scored highest on the LAION Aesthetics Predictor (LAP), indicating professional-grade visual output.
  • Scalability: The system demonstrated high efficiency, generating 1,800 unique building instances in approximately 12 hours using two NVIDIA 5090 GPUs.

Conclusion and Future Work

RAISECity represents a significant step toward automated, reality-aligned city-scale 3D generation. By leveraging a modular agentic framework, it maintains high fidelity while remaining compatible with industry-standard graphics pipelines. Future work will focus on reducing dependency on high-quality source data in sparse regions and integrating dynamic elements such as moving crowds and traffic to create truly "living" world models.

Summary updated

Abstract

City-scale 3D generation is of great importance for the development of embodied intelligence and world models. Existing methods, however, face significant challenges regarding quality, fidelity, and scalability in 3D world generation. Thus, we propose RAISECity, a \textbf{R}eality-\textbf{A}ligned \textbf{I}ntelligent \textbf{S}ynthesis \textbf{E}ngine that creates detailed, \textbf{C}ity-scale 3D worlds. We introduce an agentic framework that leverages diverse multimodal foundation tools to acquire real-world knowledge, maintain robust intermediate representations, and construct complex 3D scenes. This agentic design, featuring dynamic data processing, iterative self-reflection and refinement, and the invocation of advanced multimodal tools, minimizes cumulative errors and enhances overall performance. Extensive quantitative experiments and qualitative analyses validate the superior performance of RAISECity in real-world alignment, shape precision, texture fidelity, and aesthetics level, achieving over a 90% win-rate against existing baselines for overall perceptual quality. This combination of 3D quality, reality alignment, scalability, and seamless compatibility with computer graphics pipelines makes RAISECity a promising foundation for applications in immersive media, embodied intelligence, and world models.
BibTeX
@misc{wang2025raisecity,
  author = {Shengyuan Wang and Zhiheng Zheng and Yu Shang and Lixuan He and Yangcheng Yu and Fan Hangyu and Jie Feng and Qingmin Liao and Yong Li},
  title = {{RAISECity: A Multimodal Agent Framework for Reality-Aligned 3D World Generation at City-Scale}},
  year = {2025},
  eprint = {2511.18005},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.18005},
}
Figure