VoxScene: Anchor-Conditioned Voxel Diffusion for Indoor Scene Arrangement

Haotian Mao, Yuhan Huang, Jiatao Lin, Yang Zhao, Hui Wang, Yiheng Zhang, Yuwang Wang, Chenliang Zhou, Yan Zhang, Fangcheng Zhong, Xubo YangarXiv 2026 · arXiv

TL;DR

VoxScene is a novel 3D indoor scene synthesis framework that uses an object-centric voxel diffusion process to eliminate physical collisions and object intersections. By refining coarse layout "anchors" into explicit, non-overlapping volumetric grids, it serves as a highly effective plug-and-play generative backend that vastly improves physical plausibility and asset retrieval diversity.

What Problem Does This Paper Address?

Generative models for 3D indoor scene synthesis have come a long way, but they still suffer from a fundamental flaw: "geometric blindness." Current state-of-the-art layout planners typically rely on simple bounding boxes or implicit representations (like latent codes or global Signed Distance Fields). Because these abstractions lack a direct, native perception of 3D physical boundaries, they frequently generate layouts where furniture clips through walls, floats in mid-air, or intersects with other objects.

This spatial ambiguity becomes a major headache in densely populated environments, such as tightly arranged living rooms or complex, stacked shelf configurations. VoxScene addresses this by shifting the paradigm toward an explicit, discrete volumetric representation that naturally respects physical boundaries.

Fig. 1. VoxScene introduces a voxel-based layout representation. By leveraging the spatial exclusivity of discrete occupancies, our method resolves object intersections (shown in the red box). Furthermore, the explicit 3D structure of voxels excels in handling fine-grained layouts within complex geometric contexts (shown in the blue boxes).

Methods

VoxScene introduces an anchor-conditioned, object-centric voxel diffusion framework. Instead of attempting to generate an entire monolithic scene in a single shot, it synthesizes objects sequentially, using the following pipeline:

  1. Anchor Conditioning: The model takes coarse spatial anchors (position, orientation, size, and semantic category) from upstream sources—such as Large Language Models (LLMs), user inputs, or other data-driven layout planners.
  2. Object-Centric Local Blocks: For each target object, generation happens within a localized voxel grid block ($64^3$ resolution) centered and aligned with the object's anchor. Absolute global identifiers are remapped to relative states: free space, contextual objects, or the target object.
  3. Latent Space Compression (VQ-VAE): To make voxel diffusion computationally tractable, the authors use a 3D Vector-Quantized Variational Autoencoder (VQ-VAE) to compress the local voxel grids by a factor of 4. The network uses a dual-head design to jointly optimize discrete state channels and continuous distance fields.
  4. Latent Diffusion Model (LDM): Denoising is performed within the VQ-VAE's compact latent space using a 3D U-Net that predicts velocity targets, conditioned on dense voxel-wise tensors representing already generated context and pending object bounding boxes.
  5. Robust Training Policies: The authors implement Stochastic Context Masking (randomly dropping context objects to help the model adapt to arbitrary generation sequences) and Anchor Shifting (injecting random translations during training so the network learns to autonomously correct noisy upstream anchors).
  6. Anisotropic Asset Retrieval & Style Clustering: The synthesized high-fidelity voxels serve as geometric queries to retrieve real CAD models using a Soft Chamfer Score. A style clustering step ensures that identical furniture categories maintain visual and aesthetic consistency.
Fig. 2. Method overview. We formulate scene synthesis as an anchor-conditioned generative process. Guided by prior anchors from various upstream sources (Part 1), our object-centric framework sequentially generates each target object, completing the whole scene represented in voxels through N iterations (Part 2). These generated voxels then serve as explicit geometric proxies for downstream asset retrieval, ultimately yielding a high-fidelity scene (Part 3).

Datasets

The framework was trained and evaluated on two primary datasets:

  • 3D-FRONT: A large-scale benchmark for indoor scene synthesis, utilizing curated splits of 4,041 bedrooms, 900 dining rooms, and 813 living rooms.
  • M3D-Shelf: A customized, highly dense subset extracted from M3DLayout, consisting of 490 shelf environments containing 3 to 20 delicate objects to test fine-grained spatial reasoning.
  • 3D-FUTURE: Used as the primary CAD model database for downstream asset retrieval.

Key Insights and Ideas

  • Voxels as Hard Constraints: By leveraging the mutually exclusive nature of discrete voxels (where a single voxel can only be claimed by one object instance), the framework inherently prevents inter-object penetration at the layout level.
  • The Power of Local Context: Monolithic global scene models often blur sharp geometric boundaries and suffer from high memory overhead. Breaking the scene down into sequential, object-centric local blocks preserves fine-grained details while keeping training computationally efficient.
  • Generative Backend Refiner: Thanks to the anchor shifting strategy, VoxScene acts as a robust "collision handler." It doesn't just generate layouts from scratch; it can take messy, overlapping spatial priors from an LLM planner and cleanly untangle them into physically plausible arrangements.

Key Results

  • Physical Plausibility: On the 3D-FRONT dataset, VoxScene significantly reduced the average pairwise intersection ($I_p$) and global overlap ratio ($OR$) when paired with baselines like DiffuScene, PhyScene, and LayoutGPT. For example, adding VoxScene to DiffuScene dropped the living room overlap ratio from 8.35 to 7.18.
  • Enhanced Diversity: VoxScene successfully mitigated mode collapse, exploring a much broader range of unique CAD models and achieving substantially higher category-weighted diversity ($CD$) scores than baseline methods.
  • Efficiency in Dense Scenes: In the ultra-dense M3D-Shelf benchmark, VoxScene drastically outperformed rule-based optimization solvers like SceneWeaver. VoxScene completed shelf generation tasks in just 43.40 seconds, compared to SceneWeaver's 612.14 seconds, making it highly scalable for batch generation.

Limitations and Future Work

While VoxScene excels at local geometric refinement and fixing minor collisions, it is not designed for large-scale global scene repositioning. If upstream anchors suffer from severe errors—such as two objects completely overlapping—the model cannot fully rectify the layout.

The authors note that future iterations could introduce a rejection policy for abnormal voxels during retrieval. Another promising direction is developing an end-to-end pipeline that propagates spatial collision gradients from the retrieval algorithm directly back to the upstream layout planners.

Summary updated

Abstract

We present VoxScene, a novel anchor-conditioned voxel diffusion framework tailored for 3D scene synthesis. Current data-driven layout generation techniques typically rely on bounding proxies or implicit representations, which overlook volumetric structures. This geometric blindness inevitably leads to severe physical collisions and structural entanglement, particularly in densely populated environments. To overcome these limitations, we shift the paradigm to an explicit, object-centric voxel representation. Our pipeline sequentially synthesizes discrete volumetric occupancies conditioned on prior anchors and local context. By exploiting the mutually exclusive nature of discrete voxels, our approach eliminates spatial ambiguities and guarantees collision-free arrangements, even in highly complex environments. Furthermore, the synthesized high-fidelity voxel grids serve as discriminative geometric queries for downstream asset retrieval. Extensive experiments demonstrate the universality of our method, achieving state-of-the-art physical plausibility and unlocking shape diversity compared to existing layout planners.
BibTeX
@misc{mao2026voxscene,
  author = {Haotian Mao and Yuhan Huang and Jiatao Lin and Yang Zhao and Hui Wang and Yiheng Zhang and Yuwang Wang and Chenliang Zhou and Yan Zhang and Fangcheng Zhong and Xubo Yang},
  title = {{VoxScene: Anchor-Conditioned Voxel Diffusion for Indoor Scene Arrangement}},
  year = {2026},
  eprint = {2605.17102},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.17102},
}

Loading related content...

Figure