A Generative Model of 3D Object Layouts in Apartments

Paul Henderson, Vittorio Ferrari

Abstract

Efficient authoring of vast virtual environments hinges on algorithms that are able to automatically generate content while also being controllable. We propose a method to automatically generate furniture layouts for indoor environments. Our method is simple, efficient, human-interpretable and amenable to a wide variety of constraints. We model the composition of rooms into classes of objects and learn joint (co-occurrence) statistics from a database of training layouts. We generate new layouts by performing a sequence of conditional sampling steps, exploiting the statistics learned from the database. The generated layouts are specified as 3D object models, along with their positions and orientations. We show they are of equivalent perceived quality to the training layouts, and compare favorably to a state-of-the-art method. We incorporate constraints using a general mechanism — rejection sampling — which provides great flexibility at the cost of extra computation. We demonstrate the versatility of our method by applying a wide variety of constraints relevant to real-world applications.

Introduction

Populating virtual worlds with realistic furniture is a significant challenge for game developers, architects, and researchers. Manual placement is time-consuming and tedious, while fully automated procedural systems often produce results that lack "common sense," such as chairs facing walls or kitchens missing essential appliances. Existing solutions typically fall into two categories: optimization-based methods (like MCMC), which are high-quality but slow, and deep learning models, which are fast but often act as uncontrollable "black boxes."

This paper introduces a fast, hierarchical generative model that creates realistic 3D furniture layouts in milliseconds. By structuring the problem as a directed graphical model, the system captures the subtle logic of interior design while remaining highly controllable through user-defined constraints.

Hierarchical Generative Modeling

The core of the proposed method is a move away from heavy optimization toward ancestral sampling. Instead of solving for an entire room layout simultaneously, the model samples components in a logical, hierarchical sequence:

  1. Room Structure: The process begins by defining the room type and partitioning the space into a "cell structure" (corners, edges, and interior). This grid-based approach inherently prevents object overlaps and ensures furniture is placed in valid locations.
  2. High-Order Patterns: Rather than treating every object as an independent entity, the model learns to group them into motifs (e.g., a dining table with surrounding chairs) and abutments (e.g., a sequence of kitchen cabinets flush against a refrigerator).
  3. Embellishment: Once primary furniture is established, the model adds smaller objects like laptops, fruit baskets, and lamps based on the primary furniture's location and type.
Visualization of high-order patterns (yellow) and embellishments (blue) in a generated kitchen layout.

This structured approach allows the model to encode human design logic directly into the sampling process. Because the parameters—such as the probability of an object being against a wall—are human-readable, designers can manually tweak them to achieve specific styles.

Constraint Handling and Results

To handle user constraints (e.g., "place a bed in this specific corner"), the authors utilize rejection sampling. Because the base sampler is exceptionally fast (approximately 0.04 seconds per room), the system can generate thousands of candidate layouts and discard those that do not meet the criteria. This provides a flexible, general-purpose mechanism for imposing complex, non-differentiable constraints without the convergence issues common in optimization-based methods.

Comparison of constrained and unconstrained generation results, showing the model's ability to handle specific room shapes and object placements.

The model was trained on the SUNCG dataset, comprising over 45,000 human-designed rooms. In a perceptual "Turing test," participants were unable to distinguish between human-designed rooms and the model's unconstrained outputs, selecting the AI-generated version roughly 50% of the time. Furthermore, when compared to state-of-the-art CNN-based methods, users preferred this model's layouts in 71-75% of cases.

Conclusion

This research demonstrates that a well-structured probabilistic model can outperform complex deep learning architectures in both speed and quality for furniture layout generation. By identifying high-order patterns and leveraging the efficiency of rejection sampling, the system provides a practical tool for generating vast, realistic, and constrained virtual environments. Future work may explore more intelligent sampling techniques to handle extremely tight constraints where simple rejection sampling becomes computationally expensive.

Summary updated

Abstract

Efficient authoring of vast virtual environments hinges on algorithms that are able to automatically generate content while also being controllable. We propose a method to automatically generate furniture layouts for indoor environments. Our method is simple, efficient, human-interpretable and amenable to a wide variety of constraints. We model the composition of rooms into classes of objects and learn joint (co-occurrence) statistics from a database of training layouts. We generate new layouts by performing a sequence of conditional sampling steps, exploiting the statistics learned from the database. The generated layouts are specified as 3D object models, along with their positions and orientations. We show they are of equivalent perceived quality to the training layouts, and compare favorably to a state-of-the-art method. We incorporate constraints using a general mechanism -- rejection sampling -- which provides great flexibility at the cost of extra computation. We demonstrate the versatility of our method by applying a wide variety of constraints relevant to real-world applications.
BibTeX
@misc{henderson2017generative,
  author = {Paul Henderson and Vittorio Ferrari},
  title = {{A Generative Model of 3D Object Layouts in Apartments}},
  year = {2017},
  eprint = {1711.10939},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1711.10939},
}
Figure