Abstract
Efficient and controllable 3D scene layout generation and editing are vital for virtual reality, architectural visualization, and intelligent interaction systems. This paper introduces a generation framework that integrates a diffusion model with a spatial attention mechanism. The diffusion model ensures global layout stability through a stepwise denoising process, while the spatial attention mechanism enhances local editing accuracy by modeling object relationships. Evaluated on the 3D-FRONT and SUNCG datasets, the model achieves a layout accuracy of 89.3% and an IoU of 0.76. With inference times as low as 1.3s on GPUs, the framework offers a high-performance solution for real-time 3D content creation.
Introduction
The automated generation of realistic 3D indoor layouts—determining the optimal placement of furniture like beds, desks, and wardrobes—is a complex spatial reasoning task. While GANs and VAEs have been used, they often struggle with diversity or geometric precision. Transformer-based models offer better dependency modeling but suffer from high computational latency. Furthermore, existing methods often lack the fine-grained controllability required for interactive design, where users need to modify specific elements without disrupting the overall scene logic. This paper addresses these challenges by combining the strengths of diffusion models and attention mechanisms.
Methodology
The proposed framework employs a multi-level pipeline designed for both stability and flexibility.
Conditioned Denoising Diffusion
The model utilizes a conditioned denoising diffusion process to approximate the true distribution of scene layouts. Starting from random noise, it iteratively generates a structured layout based on user-defined constraints, such as floor plans or specific object lists. This approach prevents common issues like overlapping furniture and ensures global consistency.
Spatial Attention Mechanism
A key innovation is the spatial attention module, which dynamically weights the relationships between objects during the denoising steps. By treating furniture as nodes in a graph, the model learns semantic and geometric constraints, ensuring that functional distances (e.g., the relationship between a sofa and a coffee table) are respected.
Local Editing and Dual Representation
The system supports a "Local Editing Mode" by re-adding noise to specific parts of a scene. This allows for seamless additions, deletions, or modifications. The model processes scenes using a dual representation: voxelization for the diffusion process and graph structures for relational modeling.
Experimental Results
The model was rigorously tested on the 3D-FRONT and SUNCG datasets, along with a supplementary 300-scene dataset.
Performance Metrics
The "Diffusion + SpAttn" model achieved state-of-the-art results. Layout Accuracy (LA) was defined as correct placement within 0.20 m translation error and 15° rotation error. IoU was computed on 128³ voxel grids, and FID was calculated from five rendered views per scene using Inception-v3 features. The Editing Consistency Score (ECS) represents the ratio of satisfied spatial constraints while preserving overall structural similarity.
- Layout Accuracy (LA): 89.3%
- Intersection over Union (IoU): 0.76
- Fréchet Inception Distance (FID): 31.2
- Editing Consistency Score (ECS): 0.84
Inference Efficiency
A major focus was cross-platform viability. The model generates a scene in approximately 1.3 seconds on an NVIDIA RTX 3090 and 5.9 seconds on an NVIDIA Jetson Xavier NX, making it suitable for both high-end workstations and embedded edge devices.
Conclusion
This paper presents a robust framework for controllable 3D scene generation. By pairing diffusion models with spatial attention, the authors achieve high accuracy and editing consistency while maintaining real-time performance. Future work will explore model distillation for mobile deployment and the integration of multimodal inputs like voice and gesture commands.
Summary updated