Conditioned Denoising Diffusion with Spatial Attention for Controllable 3D Scene Layout Generation and Editing

Kaiwen Zhu, Hongyu Wu, Bin XiaoInformatica 2025 · DOI

Abstract

Efficient and controllable 3D scene layout generation and editing are vital for virtual reality, architectural visualization, and intelligent interaction systems. This paper introduces a generation framework that integrates a diffusion model with a spatial attention mechanism. The diffusion model ensures global layout stability through a stepwise denoising process, while the spatial attention mechanism enhances local editing accuracy by modeling object relationships. Evaluated on the 3D-FRONT and SUNCG datasets, the model achieves a layout accuracy of 89.3% and an IoU of 0.76. With inference times as low as 1.3s on GPUs, the framework offers a high-performance solution for real-time 3D content creation.

Introduction

The automated generation of realistic 3D indoor layouts—determining the optimal placement of furniture like beds, desks, and wardrobes—is a complex spatial reasoning task. While GANs and VAEs have been used, they often struggle with diversity or geometric precision. Transformer-based models offer better dependency modeling but suffer from high computational latency. Furthermore, existing methods often lack the fine-grained controllability required for interactive design, where users need to modify specific elements without disrupting the overall scene logic. This paper addresses these challenges by combining the strengths of diffusion models and attention mechanisms.

Methodology

The proposed framework employs a multi-level pipeline designed for both stability and flexibility.

Conditioned Denoising Diffusion

The model utilizes a conditioned denoising diffusion process to approximate the true distribution of scene layouts. Starting from random noise, it iteratively generates a structured layout based on user-defined constraints, such as floor plans or specific object lists. This approach prevents common issues like overlapping furniture and ensures global consistency.

Spatial Attention Mechanism

A key innovation is the spatial attention module, which dynamically weights the relationships between objects during the denoising steps. By treating furniture as nodes in a graph, the model learns semantic and geometric constraints, ensuring that functional distances (e.g., the relationship between a sofa and a coffee table) are respected.

The framework architecture illustrates the flow from conditional input through the diffusion and spatial attention modules to the final 3D layout.

Local Editing and Dual Representation

The system supports a "Local Editing Mode" by re-adding noise to specific parts of a scene. This allows for seamless additions, deletions, or modifications. The model processes scenes using a dual representation: voxelization for the diffusion process and graph structures for relational modeling.

Experimental Results

The model was rigorously tested on the 3D-FRONT and SUNCG datasets, along with a supplementary 300-scene dataset.

Performance Metrics

The "Diffusion + SpAttn" model achieved state-of-the-art results. Layout Accuracy (LA) was defined as correct placement within 0.20 m translation error and 15° rotation error. IoU was computed on 128³ voxel grids, and FID was calculated from five rendered views per scene using Inception-v3 features. The Editing Consistency Score (ECS) represents the ratio of satisfied spatial constraints while preserving overall structural similarity.

  • Layout Accuracy (LA): 89.3%
  • Intersection over Union (IoU): 0.76
  • Fréchet Inception Distance (FID): 31.2
  • Editing Consistency Score (ECS): 0.84
Ablation study demonstrating the performance gains achieved by integrating conditional control and spatial attention into the diffusion framework.

Inference Efficiency

A major focus was cross-platform viability. The model generates a scene in approximately 1.3 seconds on an NVIDIA RTX 3090 and 5.9 seconds on an NVIDIA Jetson Xavier NX, making it suitable for both high-end workstations and embedded edge devices.

Conclusion

This paper presents a robust framework for controllable 3D scene generation. By pairing diffusion models with spatial attention, the authors achieve high accuracy and editing consistency while maintaining real-time performance. Future work will explore model distillation for mobile deployment and the integration of multimodal inputs like voice and gesture commands.

Summary updated

Abstract

Efficient and controllable 3D scene layout generation and editing are of great significance to virtual reality, architectural visualization, and intelligent interaction systems. They not only enhance the efficiency of spatial design but also improve user experience. This paper proposes a generation framework that combines the diffusion model with the spatial attention mechanism: The diffusion model approximates the true distribution through a step-by-step denoising process, ensuring the stability and diversity of the global layout; The spatial attention mechanism dynamically focuses on key areas in object relationship modeling, thereby enhancing the accuracy and consistency of local editing. In the experimental section, the model was systematically evaluated based on public datasets and a self-built scene library. Performance metrics such as layout accuracy (89.3%), intersection over union (IoU) (0.76), Fréchet Inception Distance (FID) (31.2), and editing consistency score (0.84) were used for performance measurement. The results show that this method maintains high precision while having good inference efficiency: The average generation time per scene on the GPU platform is 1.3 s, and about 5.9 s on embedded devices, which is superior to baseline methods. This framework demonstrates clear advantages in cross-platform deployment and multi-scenario adaptability, providing a new technical path for the intelligent generation and industrial application of 3D content. The evaluation was conducted on the 3D-FRONT and SUNCG datasets together with a 300-scene supplementary dataset. Layout Accuracy was defined as correct placement within 0.20 m translation error and 15 ° rotation error., IoU was computed on 128³ voxel grids, FID was calculated from five rendered views per scene using Inception-v3 features, and the Editing Consistency Score was defined as the ratio of satisfied spatial constraints while preserving overall structural similarity.
BibTeX
@inproceedings{zhu2025conditioned,
  author = {Kaiwen Zhu and Hongyu Wu and Bin Xiao},
  title = {{Conditioned Denoising Diffusion with Spatial Attention for Controllable 3D Scene Layout Generation and Editing}},
  booktitle = {Informatica},
  year = {2025},
  doi = {10.31449/inf.v49i14.11284},
}
Figure