Text2City: One-Stage Text-Driven Urban Layout Regeneration

Yiming Qin, Nanxuan Zhao, Bin Sheng, Rynson W. H. LauAAAI 2024 · DOI

Introduction

Urban layout design is a complex task traditionally reserved for experts using procedural tools like CityEngine. These tools require intricate rule-setting and lack intuitive interfaces for non-specialists. Text2City addresses this by introducing a one-stage framework that regenerates urban layouts—specifically roads and buildings—from simple natural language descriptions. By leveraging diffusion models and a unique "knowledge exchange" module, it ensures that generated cityscapes are not only visually realistic but also functionally rational.

Text2City generates structured urban layouts consisting of roads and buildings.

The Problem with Sequential Generation

Existing deep learning approaches often follow a two-stage pipeline: generating a road network first and then populating it with buildings. This sequential method fails to capture the inherent interaction between these elements. For instance, buildings should align with roads, and roads should respect existing structures. Furthermore, most models lack support for intuitive text-based inputs, making it difficult to specify nuanced requirements like "a residential area with narrow pedestrian paths."

The Text2City Architecture

Text2City utilizes Conditioned Denoising Diffusion Probabilistic Models (DDPMs) to generate roads and buildings simultaneously. The framework is built on three core pillars:

  • Text-to-Attribute (T2A) Network: Leveraging Sentence-BERT and CLIP, this module decouples user prompts into specific attributes for roads (width, hierarchy) and buildings (density, function).
  • Joint Regenerator Network: This network processes two parallel streams for roads and buildings, denoising them from random noise into a structured layout in a single stage.
  • Interactive & Enhanced Guidance (IEG) Module: This is the core innovation. It facilitates "prior knowledge exchange" between the road and building streams. By allowing the streams to "talk" to each other, the model ensures that roads lead to buildings without overlapping them, resulting in a harmonious cityscape.
The model allows for targeted regeneration of urban regions based on text prompts.

To maintain geometric rationality, the authors implement multi-level constraints, including an Isoperimetric Quotient loss to ensure buildings maintain compact, realistic shapes rather than irregular blobs.

Data and Performance

The researchers developed the Text2City Dataset, featuring 147,000 regions from Greater London OpenStreetMap data, complete with layered image data and textual annotations. Testing on cities like Paris and Shanghai demonstrated the model's strong generalization capabilities.

Quantitatively, Text2City outperformed state-of-the-art models like Stable Diffusion (SDv2), achieving a Structural Similarity (SSIM) of 0.89 (vs. 0.72 for SDv2) and significantly lower FID scores. In user studies, participants consistently rated Text2City as the most "rational" and "harmonious" model, effectively translating text descriptions into functional urban patterns.

Future Outlook

While Text2City marks a significant step forward, challenges remain. Inference speed is currently a bottleneck for real-time design, and the model lacks fine-grained control over individual building placement. Future work aims to extend the framework to 3D layouts and incorporate additional semantic layers like green spaces and street furniture.

Summary updated

BibTeX
@inproceedings{qin2024text,
  author = {Yiming Qin and Nanxuan Zhao and Bin Sheng and Rynson W. H. Lau},
  title = {{Text2City: One-Stage Text-Driven Urban Layout Regeneration}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i5.28257},
}
Figure