Ask a system to recover the walls, floor and ceiling of a furnished room from one 360° photo and it runs into a basic problem. The corners and wall–floor seams that define the room are often hidden behind wardrobes, sofas and shelving. A single image is also consistent with many different 3D rooms. Current methods take one of two routes. Some predict depth for every pixel and then fit a box or polygon to it afterwards. Others predict boundaries in 2D image space or on a flattened top-down view. In both cases, once the seams themselves are invisible, the method has nothing left to anchor on. We propose a third route. The network predicts a small set of 3D planes directly, each one forced by construction to obey the "Atlanta world" rule. That rule says floors and ceilings are horizontal and walls are vertical, but walls may face any compass direction, so the network does not assume right-angle (Manhattan) rooms. A hidden corner is then computed as the intersection of two walls that are each partly visible elsewhere in the image, rather than being looked for where it cannot be seen.
The central claim can be tested: parameterizing the room shell as gravity-constrained plane queries should make layout accuracy degrade more slowly as clutter increases than either per-pixel amodal depth followed by plane fitting or 2D raster layout prediction. Here "amodal" depth means depth to the walls behind the furniture, not to the furniture itself. We make the claim measurable with a controlled clutter axis. Structured3D renders the same room empty, simply furnished and fully furnished, so we can trace an error-versus-occlusion curve for every method on identical geometry.
What exists and where the gap is
Three lines of work each supply part of this idea, and none combines all of it.
The first line produces amodal room geometry under clutter, but through dense intermediate maps. Joint 3D Layout and Depth Prediction from a Single Indoor Panorama Image predicts a layout depth map of the empty room envelope and then fits a strict Manhattan model as post-processing, reaching 85.81% 3D IoU on Stanford 2D-3D. Its authors report that it degrades when clutter covers most wall surfaces. Geometric Structure Based and Regularized Depth Estimation From 360 Indoor Imagery adds a furniture attention mask and a Manhattan planarity penalty, and acknowledges that it struggles with non-orthogonal rooms. NadirFloorNet predicts uncluttered depth, projects it onto a top-down plane and assembles room polygons, but assumes a single ceiling height per room. These papers motivate the proposal: amodal supervision from paired empty and furnished renders works. They also constrain it: in all three, the structural prior arrives after a dense prediction that is itself unreliable where occlusion is heaviest.
The second line relaxes the Manhattan assumption. AtlantaNet projects the panorama onto ceiling and floor views and decodes 2D contours, reaching 84.26% 3D IoU on the non-Manhattan AtlantaLayout set. It still outputs 2D projections and relies on the ceiling being visible. AtlasGS and Neural 3D Scene Reconstruction with the Manhattan-world Assumption impose Atlanta and Manhattan regularizers through a loss during multi-view optimization. Both need dense views and a 2D semantic segmenter, so segmentation errors become geometry errors. We take the Atlanta model from this line but enforce it differently: through the output parameterization of a single-view feed-forward network, with no segmenter in the loop.
The third line supplies the mechanism. Unified Primitive Proxies for Structured Shape Completion (UniCo) uses learnable queries that each decode one geometric primitive with a confidence score, matched to ground truth during training. Replacing a completion-then-RANSAC cascade this way cut Chamfer distance by up to 50% on partial CAD point clouds. UniCo itself names scaling to indoor scenes as open, and it works only on point clouds of isolated objects. GeoLayout is the nearest relative in the layout literature. It predicts per-pixel plane parameters, clusters them into walls, and intersects the clusters to get layout. It lacks any gravity or Atlanta constraint, and its clustering can mistake a flat wardrobe front for a wall. Layout Anything works end to end but in 2D raster space, and reports under-segmentation when large furniture hides junctions. Its "degeneration" augmentation, which removes layout surfaces during training, is a cheap form of occlusion robustness that we must control for.
Core Claim
Each query in our model predicts one gravity-aligned plane. A horizontal plane is a single height. A wall is an azimuth angle, a signed distance from the camera, and a horizontal extent along its own direction. Every query also outputs a probability that its plane exists in the room. Because verticality and horizontality are built into the output, the network cannot produce a tilted wall even when the image evidence near a sofa is ambiguous. Because each wall is a global object, a wall seen only above a bookcase still pins down the hidden corner where it meets its neighbour. We expect the advantage to grow with the fraction of structural seams hidden by furniture and to be negligible in empty rooms. If the advantage is uniform across clutter levels, or absent, the mechanism story is wrong.
Research Program
The model encodes an equirectangular panorama with a standard backbone. A set of learnable queries (around 24, more than any room in the training data has planes) attends to the image features through a transformer decoder, as in UniCo and DETR-style detectors. Each query decodes into one of three classes: floor-type horizontal plane, ceiling-type horizontal plane, or wall. Multiple floor and ceiling planes are allowed, so stepped ceilings, which defeat NadirFloorNet's single-height assumption, are representable.
Training uses Structured3D, which provides exact plane primitives and paired renders. The input is always the "full" or "simple" furnished render. The targets are the planes of the empty room. Queries are matched to ground-truth planes by minimum-cost assignment on class, parameters and extent, with UniCo's online reassignment to stabilize early training.
A second loss renders the predicted plane set into an amodal depth panorama: each pixel takes the nearest ray–plane intersection that lies within a wall's extent, using a soft minimum so gradients flow. This rendered depth is compared against the empty-room depth. The rendering loss is what teaches walls to close into a watertight shell instead of floating as independent detections. The final layout is read off by intersecting adjacent walls in the top-down plane and extruding between floor and ceiling heights. No box or polygon fitting happens afterwards.
For real data, we fine-tune on MatterportLayout and ZInD layout annotations through the same rendered-boundary path. The 100-panorama AtlantaLayout set is held out entirely as a non-Manhattan test.
The analysis component is the clutter-robustness curve. On Structured3D we evaluate every method on the same rooms under empty, simple and full furniture. On real panoramas with sensor depth (Stanford 2D-3D and Matterport), we define an occlusion score for each image: the fraction of ground-truth layout-boundary pixels where measured depth is clearly shorter than layout depth, meaning something stands in front of the seam. Results are binned by this score. We also check whether each wall's confidence is lower when that wall is mostly hidden. This tests whether the model knows the difference between walls it saw and walls it inferred, which is our concession to the problem being ill-posed.
Baselines
The naive reference is a per-pixel amodal depth network with the same backbone, trained on the same Structured3D pairs, whose output is converted to layout by taking the farthest-surface contour in the top-down view without any plane structure. This shows what the paired data alone buys. A second naive point is the same pipeline with a strict Manhattan box fit, which shows the cost of the orthogonality assumption on AtlantaLayout.
The strongest structured competitor is the cascade that a skeptic would build: the same amodal depth network, followed by Atlanta-constrained RANSAC (vertical walls with free azimuth, horizontal floor and ceiling) and polygon closing. This is the Zeng et al. recipe with Manhattan fitting replaced by Atlanta fitting. It receives the same backbone, data, training budget and a held-out tuning sweep over RANSAC thresholds. Beating it means a higher 3D IoU in the top occlusion tercile, with non-overlapping confidence intervals across three seeds, while matching it within noise on empty rooms.
Among published methods, AtlantaNet is the strongest on non-Manhattan rooms. It reports 84.26% 3D IoU and 88.78% 2D IoU on AtlantaLayout (AtlantaNet). Bi-Layout reports 82.57% 3D IoU on MatterportLayout, and Zeng et al. report 85.81% 3D IoU on Stanford 2D-3D. We will retrain AtlantaNet with Structured3D pretraining so that data is not the difference. Structured3D itself showed that such pretraining lifts HorizonNet from 83.42% to 85.27% 3D IoU on PanoContext. Beating AtlantaNet on its own benchmark would be a positive signal, but the main comparison is the occlusion-binned one.
Layout Anything is a perspective-image method with no reported panorama result on Lacuna. We include a panorama adaptation of its raster formulation, with its degeneration augmentation, as the end-to-end 2D competitor.
Three ablations isolate the key ingredient. The first keeps the queries but removes the gravity constraint, predicting free 3D planes. The second keeps the queries but imposes Manhattan orthogonality. The third removes the rendering loss.
Why not just…?
Why not just predict amodal depth and fit Atlanta planes afterwards? This is the real threat. UniCo's evidence against cascades comes from CAD point clouds, not rooms, so it does not settle the question here. Our expectation is that the fit gets few reliable inliers exactly where walls are hidden, because per-pixel depth behind furniture is hallucinated independently at each pixel. In light clutter the cascade may well match us. The matched-cascade baseline and the occlusion-binned curve decide this. If the cascade is within one 3D IoU point in the heaviest bin, we will report that the plane-query parameterization is unnecessary. The contribution then shrinks to the Atlanta cascade recipe and the clutter benchmark protocol.
Why not just train an existing layout network on Structured3D's furnished renders? Paired data is a large part of the gain, as the HorizonNet pretraining result shows. That is why AtlantaNet and the raster baseline are retrained on identical data. If they close the gap, then representation is not the bottleneck.
Why not just remove the furniture with inpainting, then run a layout estimator? Seeing Through Clutter does this with a chain of foundation models. It applies no architectural constraint, and it reports inpainting artifacts that need heuristic filtering. Lacuna lists no layout-benchmark numbers for it. We will run an inpaint-then-AtlantaNet pipeline on the occlusion-binned subsets. We expect inpainted walls behind large furniture to be visually plausible but geometrically inconsistent, which would show up as errors in corner position.
Why not just use the ceiling, which furniture rarely blocks? AtlantaNet's own mixed-feeding result shows that ceiling-only training is insufficient. Stepped ceilings break the floor–ceiling mirror that the ceiling trick relies on. Our multi-height horizontal planes target this case, and AtlantaNet's ceiling branch is part of the published baseline.
Proposed Experiments
The decisive experiment is the occlusion-binned comparison between plane queries and the matched Atlanta cascade, on Structured3D paired renders and on real Stanford 2D-3D and MatterportLayout panoramas. Metrics are 3D IoU, 2D IoU and corner error, plus amodal depth error restricted to pixels that are occluded in the furnished render.
Generalization is tested on held-out AtlantaLayout and on ZInD. Calibration is measured as the expected calibration error of wall-existence confidence, reported separately for visible and hidden walls.
Two results falsify the claim. The first is that the gap over the cascade does not widen with occlusion. The second is that the unconstrained-plane ablation matches the Atlanta version, which would mean the gravity constraint is not doing the work.
References
- Joint 3D Layout and Depth Prediction from a Single Indoor Panorama Image
- Geometric Structure Based and Regularized Depth Estimation From 360 Indoor Imagery
- NadirFloorNet: reconstructing multi-room floorplans from a small set of registered panoramic images
- AtlantaNet: Inferring the 3D Indoor Layout from a Single 360(∘) Image beyond the Manhattan World Assumption
- AtlasGS: Atlanta-world Guided Surface Reconstruction with Implicit Structured Gaussians
- Neural 3D Scene Reconstruction with the Manhattan-world Assumption
- Unified Primitive Proxies for Structured Shape Completion
- GeoLayout: Geometry Driven Room Layout Estimation Based on Depth Maps of Planes
- Layout Anything: One Transformer for Universal Room Layout Estimation
- No More Ambiguity in 360° Room Layout via Bi-Layout Estimation
- Structured3D: A Large Photo-Realistic Dataset for Structured 3D Modeling
- Seeing Through Clutter: Structured 3D Scene Reconstruction via Iterative Object Removal