TL;DR
FurniScene introduces a massive new 3D indoor scene dataset featuring over 111,000 rooms meticulously detailed with both large furniture and tiny everyday decorations (like cups, books, and vases). To handle this unprecedented level of detail, the authors propose a Two-Stage Diffusion Scene Model (TSDSM) that separates object selection from spatial placement, significantly advancing the realism and diversity of automated interior design.
What Problem Does This Paper Address?
Generative AI has made massive strides in creating realistic images and text, but generating realistic 3D indoor spaces remains a stubborn challenge. While current models can arrange large furniture items (like beds and sofas) into a reasonable layout, the resulting rooms often feel sterile, monotonous, and distinctly unlivable.
The root of the problem is twofold:
- Dataset Limitations: Existing 3D room datasets lack "clutter"—the small, everyday objects like coffee cups, books, and flowerpots that make a house feel like a home.
- Optimization Overload: Current generation algorithms try to decide what objects belong in a room and where they should go all at the same time. When a scene contains dozens of tiny decorative items, this single-stage approach becomes too complex, leading to chaotic layouts, missing pieces, or objects awkwardly clipping into one another.
Methods
To solve the optimization challenge, the authors developed the Two-Stage Diffusion Scene Model (TSDSM). Instead of generating a fully furnished room in one shot, TSDSM breaks the task into a logical pipeline:
- Stage 1: Furniture List Generation Model (FLGM): Given a text prompt, a diffusion model predicts an unordered list of the categories and sizes of all objects that should be in the room. Crucially, the authors found that treating this as an unordered set (and omitting positional encodings) yielded the best results.
- Stage 2: Layout Generation Model (LGM): A Furniture Retrieval System (FRS) matches the predicted list with actual CAD models from the database. Then, a second diffusion model (utilizing a 1D-UNet architecture) takes these fixed objects and determines their precise 3D locations and rotations.
To prevent objects from overlapping, the layout model incorporates a specialized bounding box intersection loss ($L_{box}$) based on Intersection over Union (IoU) metrics during training.
Datasets
The core contribution of this paper is FurniScene, a large-scale synthetic 3D room dataset curated by professional interior designers.
- Scale: It contains 111,698 rooms and 39,691 unique furniture CAD models.
- Diversity: It spans 15 distinct room types and 89 object classes—including 40 categories dedicated purely to small decorative items.
- Density: Rooms average 14.4 objects (with a maximum of 119), vastly outstripping older datasets like 3D-FRONT (which averages 6.9 objects per room).
- Formats: The dataset provides high-resolution textures, rich geometric details, and a semantic 3D point cloud version (evenly sampling 30,000 points per furniture piece) to support 3D semantic segmentation research.
The data pipeline involves importing professional SketchUp designs, refining them in 3DMax for precise mesh segmentation, annotating them in Unreal Engine, and applying data augmentations (rotation, probabilistic deletion, and replacement).
Key Insights and Ideas
- The "Clutter" Crux: Realism in virtual scenes is a long-tail problem. True realism isn't achieved by just placing a bed; it's achieved by placing the lamp, the alarm clock, and the book next to it.
- Divide and Conquer: Locking the object categories and sizes in the first stage acts as a powerful prior for the second stage. Because the layout network doesn't have to guess how many items to place, it can focus entirely on spatial harmony.
- Single-Step Failure Modes: Existing single-stage models like DiffuScene struggle with FurniScene's complexity, frequently generating rooms with either an excessive or insufficient number of objects because the network gets overwhelmed by the sheer number of attributes it has to denoise simultaneously.
Key Results
The authors benchmarked TSDSM against leading models (SceneFormer, ATISS, and DiffuScene) across bedrooms, living rooms, and dining rooms using metrics like Frechet Inception Distance (FID) and Scene Classification Accuracy (SCA).
- Superior Realism: TSDSM significantly outperformed all baselines. In living room generation, TSDSM achieved an FID score of 36.54 (lower is better), compared to ATISS (63.63) and DiffuScene (74.31).
- Plausible Distributions: The model scored exceptionally well on Category KL Divergence (CKL), proving that the generated object distributions closely mirror real-world interior design distributions.
- Visual Quality: Qualitative evaluations showed that TSDSM successfully populates rooms with realistic amounts of decor without the severe object overlapping issues that plague autoregressive models like SceneFormer.
Limitations and Future Work
While TSDSM represents a major step forward, the authors note that object positions and orientations are still "not optimal" in every scenario. Fine-grained spatial relationships—such as ensuring a teacup sits precisely on a saucer rather than hovering slightly or clipping into a table—remain a challenge.
Additionally, the initial dataset creation pipeline is incredibly labor-intensive, requiring professional 3D modelers to spend hours manually aligning and segmenting assets before data augmentation can take place. Future work will likely target automated asset cleaning and even tighter spatial constraints during the layout phase.
Summary updated