Abstract
3D shape generation techniques utilizing deep learning are increasing attention from both computer vision and architectural design. This survey focuses on investigating and comparing the current latest approaches to 3D object generation with deep generative models (DGMs), including Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), 3D-aware images, and diffusion models. We discuss 187 articles (80.7% of articles published between 2018-2022) to review the field of generated possibilities of architecture in virtual environments, limited to the architecture form. We provide an overview of architectural research, virtual environment, and related technical approaches, followed by a review of recent trends in discrete voxel generation, 3D models generated from 2D images, and conditional parameters. We highlight under-explored issues in 3D generation and parameterized control that is worth further investigation. Moreover, we speculate that four research agendas including data limitation, editability, evaluation metrics, and human-computer interaction are important enablers of ubiquitous interaction with immersive systems in architecture for computer-aided design. Our work contributes to researchers' understanding of the current potential and future needs of deep learnings in generating virtual architecture.
Introduction: The Paradigm Shift to Virtual Liberty
Traditional architecture has long been bound by the constraints of "reinforced concrete"—the physical laws of gravity, safety regulations, and economic realities. However, as we transition into the metaverse, a new paradigm emerges: "AI-Architecture Liberty." This concept envisions the design of complex 3D environments free from material costs and physical limitations.
Despite this potential, virtual architecture often remains stagnant, either mimicking real-world buildings or relying on labor-intensive manual modeling. A significant "knowledge gap" exists between the computer vision (CV) community, which develops powerful 3D shape generation algorithms, and the architectural design community, which requires tools that respect design intuition and user experience. This survey systematizes the current state of Deep Generative Models (DGMs) to bridge this gap.
Methodological Evolution in AI-Assisted Design
The survey categorizes the evolution of AI-assisted design into three distinct methodological shifts:
- 3D Transposition (2D to 3D): Early methods utilized 2D generative models, such as StyleGAN or Pix2Pix, to create floor plans or facades. These were then manually or procedurally extruded into 3D. While providing high resolution, these methods often lack direct geometric control.
- 3D Solid Generation: This stage involves models that "think" natively in 3D. The research analyzes the use of Generative Adversarial Networks (GANs) for voxel grids and point clouds, alongside Variational Autoencoders (VAEs) for manipulating design features within a latent space.
- Text-to-3D and Multimodal Synthesis: The current frontier leverages Diffusion Models and Neural Radiance Fields (NeRFs). These allow designers to generate complex 3D forms using natural language prompts (e.g., "a bionic skyscraper inspired by sea shells") or by combining text with reference images.
The Role of Data in Virtual Construction
In the realm of AI architecture, "data is the new concrete." The survey highlights several foundational datasets:
- BuildingNet: A massive collection of 3D building models with detailed semantic labels for exteriors.
- HouseNet: A curated dataset specifically for residential architectural forms.
- 3DBAG: A specialized dataset containing 3D models of millions of buildings in the Netherlands, used for training models on urban morphology.
- ShapeNet: A foundational benchmark for general 3D object generation.
The lack of unified, high-quality, and architecturally "meaningful" 3D datasets remains the primary bottleneck for the field.
Key Insights: Agency and Interaction
Virtual architecture is evolving from static shells into integral parts of a data network. Buildings in the metaverse can be "dramaturgical," changing shape or function based on social interaction or real-time data. This shift necessitates a move from focusing on how a building looks to how it behaves—a field known as Human-Building Interaction (HBI).
Furthermore, the goal of these tools is not to replace the architect but to establish "Agency." AI should act as a collaborative partner that understands "design intuition"—the subtle, non-verbal logic that guides human creators.
Trends and Results
The field has seen explosive growth in recent years. The survey found that over 81% of relevant research was published between 2019 and 2023, signaling a massive spike in interest in AI-generated 3D content. While GANs were the dominant tool until 2021, Diffusion Models have rapidly become the state-of-the-art for generating high-fidelity, text-guided architectural forms.
Challenges and Future Directions
Several hurdles remain before AI-Architecture Liberty is fully realized:
- The "Black Box" Problem: Most current models lack explainability, making it difficult for designers to tweak specific parameters precisely.
- Inclusivity and Bias: Models trained primarily on Western architectural datasets risk stifling global design diversity.
- Ethics and Ownership: As AI generates more content, questions of copyright and architectural authorship become urgent legal challenges.
Future research must focus on Human-AI Collaboration, specifically developing tools that allow for real-time, "conversational" design between the human architect and the generative model.
Summary updated