Topology as the Glue: Refining Floorplans Assembled from Sparse Panoramas

When a home is captured with a handful of 360° photos, current systems build the floorplan like a jigsaw puzzle. They estimate each room's outline from one panorama, guess how rooms fit together by matching doors and windows seen from both sides, and then stitch the pieces into a house. The stitching step is where plans quietly break. Neighbouring rooms leave thin slivers of empty space between walls that should touch, overlap where they should abut, or fall into disconnected islands. These plans look nearly right by area overlap and are useless as floorplans. The usual remedy is the Manhattan world assumption, the prior that every wall runs along one of two perpendicular directions. Snapping walls to that grid closes gaps as a side effect, and it distorts exactly the angled rooms that make real homes hard.

We propose to replace that orthogonality prior with a topological one. Render the whole assembled house into an overhead image through a differentiable rasterizer, meaning a renderer whose output pixels have useful gradients with respect to polygon vertex positions. Then penalize short-lived topological features of that image, such as tiny holes and stray pieces, that are the signature of stitching errors. Those gradients flow back jointly into room poses and wall vertices. A door-to-door semantic alignment term keeps the refinement anchored to what the cameras actually saw. The scale at which a hole counts as a "gap" is set by wall thickness, not by wall angle, so the prior should close gaps equally well in rectilinear and non-rectilinear rooms. Whether it does is the central empirical question.

Core Claim

The fragmentation that matters in panorama-based floorplan reconstruction lives at the level of the assembled house, not the single room. A single room polygon is closed by construction. The defects appear between rooms. We claim these defects can be detected without ground truth and without knowing which walls should be adjacent. They show up as low-persistence features in the persistent homology of the rasterized union of room masks. Persistent homology is a tool that tracks connected pieces and holes as a threshold sweeps over an image; a feature's "persistence" is how long it survives that sweep.

A sliver gap between two rooms is a hole a few centimetres wide, so it has low persistence. A legitimate uncaptured closet or courtyard is a large hole with high persistence. Penalizing only the former, through a differentiable rasterizer, gives a label-free regularizer that closes the plan without imposing right angles. The residual topological defect left after refinement also serves as a second signal for accepting or rejecting a room-to-room alignment guess.

What Exists and What Is Missing

The closest system is SALVe. It matches windows, doors and openings across sparse ZInD panoramas and verifies each pairwise alignment guess with a convolutional network applied to overhead floor and ceiling texture patches. It then optimizes a pose graph and stitches room polygons heuristically. SALVe motivates this proposal and supplies its front end. It shows semantic anchors beat classical structure-from-motion, reporting 57.1% of panoramas localized versus 22.2% for OpenSfM and mean 2D IoU of 0.49 versus 0.29. Its stitching, however, is non-differentiable, and it never checks the topology of the result. We keep its alignment guesses and replace the heuristic stitch with gradient-based joint refinement.

Two point-cloud floorplan methods constrain the design. CAGE represents rooms as directed wall edges and closes loops by deterministic intersection. PolyRoom represents each room as a fixed-length vertex sequence and uses separate attention within rooms and between rooms. Both show that closing individual room loops can be solved representationally. That is why we do not spend effort on per-room closure and target the relationships between rooms instead. Both also take clean density maps from 3D scans, not unposed panoramas.

The rasterization machinery is reused, not invented. MapVR renders polygons as soft masks via signed distance functions so that pixel Dice loss trains vector outputs, gaining 3.5–5.9 mAP on nuScenes. Soft Rasterizer supplies the general soft-coverage formulation. MapVR adds no topological term, and its lanes can still fracture.

The topological loss comes from TopoMatch and its efficient successor Efficient Betti Matching. They show that overlap losses ignore topology and that naive persistence matching pairs features in the wrong places, reporting spatially incorrect matches "up to 99.4% of the time." They fix this with induced matchings on pixel likelihood maps. Their work is supervised and pixel-only. Our main term needs no ground truth, because it only penalizes short-lived features. Betti matching is used only in a supervised variant, where it is applied to rendered vector geometry rather than to network logits.

On the Manhattan side, A Mixture of Manhattan Frames and AtlasGS relax orthogonality to several axis sets. They are the natural competitor, a softer orientation prior, and we treat them as such. ZInD quantifies the stakes. Roughly 12.5% of its rooms are non-Manhattan, and layout 3D IoU drops from 88.2% on cuboid rooms to about 81% on non-Manhattan and non-flat-ceiling rooms.

Research Program

The input is SALVe's output on a ZInD floor: per-panorama room polygons from its layout estimator, detected window/door/opening segments, and a globally optimized set of room poses. The variables we refine are each room's rigid pose (2D translation and rotation) and a small residual offset on each wall vertex, with the offsets kept small by an L2 penalty.

Every room is rasterized into a shared overhead grid at roughly 2 cm per pixel using a soft signed-distance polygon renderer in the style of MapVR. The house mask $U$ is the soft union of the room masks. The objective is

$$ \mathcal{L} = \mathcal{L}_{\text{anchor}} + \lambda_{o}\sum_{i<j} \langle M_i, M_j \rangle + \lambda_{t}\Big[\sum_{f \in H_1(U),\, p_f<\tau} p_f^2 + \sum_{f \in H_0(U),\, f \neq f^*} p_f^2\Big]. $$

Here $M_i$ is the soft mask of room $i$, and the inner product measures overlapping area. $\mathcal{L}_{\text{anchor}}$ is a soft-line Dice loss that makes each matched door or opening in room $i$ coincide with its partner in room $j$. $H_0(U)$ and $H_1(U)$ are the connected-component and hole features of the cubical persistence of $U$, and $p_f$ is the persistence of feature $f$. $f^*$ is the most persistent component, meaning the house itself. $\tau$ is a persistence threshold tied to maximum interior wall thickness, which we set from ZInD statistics near 30 cm and sweep in experiments.

Gradients of the persistence terms reach vertices through the critical pixels, following TopoMatch-style backpropagation. The overhead grid of a single floor is small enough that cubical persistence is affordable at each of a few hundred optimization steps. We will report wall-clock time, since the known cubic worst case is the main cost risk.

The second contribution reuses the refined objective as a verifier. For a candidate alignment that SALVe's classifier scores near its decision boundary, we run the refinement with and without that edge. The change in the topological residual becomes an extra feature for accepting or rejecting the edge. This is where completeness gains could come from: floors that fragmented into disconnected components could be reconnected by edges the classifier was unsure about.

A supervised variant trains a small network to predict the vertex offsets directly, using Betti matching against ground-truth ZInD floorplans rendered through the same rasterizer. This tests whether amortizing the refinement helps, but it is secondary.

Baselines

The do-nothing reference is SALVe's own output with its heuristic stitching, run from the released pipeline on the ZInD test split. Its reported numbers are 57.1% completeness, mean 2D IoU of 0.49 and median rotation error of 0.17°, from the SALVe paper. OpenSfM, at 22.2% and 0.29, serves as a lower anchor. Every refinement method starts from exactly the same SALVe poses, polygons and anchor matches, so differences reflect only the refinement stage.

The strongest competing prior is orientation snapping. We will implement it with the same rasterizer and anchor loss, replacing the topology term with an energy that pulls each wall toward the nearest dominant direction of a per-house mixture of Manhattan frames, following the mixture-of-frames formulation. We will also run a strict single-frame Manhattan version. Its weight gets the same tuning budget on a validation split as our $\lambda_t$ and $\tau$. It is the strongest competitor because snapping is exactly how the field currently closes gaps. Beating it means matching it on Manhattan-only floors while clearly exceeding it on floors that contain non-Manhattan rooms.

The strongest non-topological competitor is a direct geometric gap penalty. For each pair of rooms with a matched door, it pulls every wall segment of one room toward the nearest parallel wall segment of the other, within the same distance threshold $\tau$. Overlap and anchor terms stay identical. This baseline isolates whether persistent homology contributes anything beyond "walls near each other should touch."

Ablations remove, one at a time, the $H_1$ term, the $H_0$ term, the overlap term, and the anchor term. A final ablation varies $\tau$ to check that performance depends on wall-thickness scale rather than on a lucky setting.

Beating a baseline means one of two things, on held-out floors with bootstrap confidence intervals over homes. Either sliver and overlap defects fall with no loss in 2D IoU, or completeness rises when the verifier feature is added.

Why not just…?

Why not just snap walls to Manhattan or Atlanta frames? On ZInD, where most rooms are rectilinear, this is a real threat, and it may win overall on the Manhattan majority. ZInD's own drop from 88.2% to about 81% IoU on non-Manhattan layouts shows the cost of orthogonality priors on the remaining rooms. The stratified comparison in our experiments settles this directly. If snapping matches us on non-Manhattan floors too, the topological prior is not justified.

Why not just penalize distances between nearby walls directly, without persistent homology? This is the most serious alternative, and we treat it as a co-equal baseline rather than a strawman. Our argument is that a direct wall-to-wall penalty needs a guess about which walls are adjacent. With a threshold it will also pull together walls that bound a real narrow closet or hallway. Persistence instead measures hole width over the whole union without any correspondences. Whether that distinction matters in practice is empirical. If the geometric penalty matches our defect and IoU numbers, we will report that the topology term buys nothing beyond it. We will then retain only the verifier contribution, which a local wall penalty cannot supply.

Why not just clean up the raster afterwards with morphological closing? Closing followed by re-vectorization fills slivers in the image. It does not move room poses, so errors at doors persist. It also re-introduces the thresholding and polygon-extraction fragility the dossier notes for raster-only pipelines. We include it as a post-hoc reference in the defect metrics. We expect it to reduce sliver counts while leaving pose error and door alignment unchanged.

Why not just train a better pairwise verifier? A better classifier helps edge selection. However, SALVe's errors after pose-graph optimization include slightly misplaced but correctly accepted rooms, which no classifier fixes. The ablation that keeps refinement but drops the verifier feature, and its converse, separates these two effects.

Proposed Experiments

The primary benchmark is the ZInD test split, stratified into floors with and without non-Manhattan rooms using ZInD annotations. We report SALVe's metrics (completeness, mean 2D IoU, median rotation and translation error), because the file gives no reported topology metric for any floorplan method on ZInD. We add three topology metrics of our own. The first is the fraction of floors whose rasterized union is one component with no hole of persistence below 30 cm. The second is total inter-room overlap area. The third is Betti-number error of the house free space against the ground-truth floorplan.

A controlled experiment uses Structured3D panorama renders. Starting from ground-truth room polygons, we inject known pose and vertex perturbations of increasing magnitude. This measures the basin within which each method recovers a closed plan, and it tests the known risk that soft rasterizers lose gradients far from boundaries. In that regime we will use a coarse-to-fine mask blur.

The decisive checks are three. First, on the non-Manhattan stratum, the topology term must reduce sliver and overlap defects relative to mixture-of-Manhattan snapping at equal or better 2D IoU. Second, relative to the direct geometric gap penalty, it must win on floors containing narrow legitimate spaces. We will identify those floors in advance from ground-truth rooms narrower than 1.2 m, since that is where correspondence-free persistence should matter. Third, adding the topological residual as a verifier feature must raise completeness above SALVe's 57.1%.

If the first check fails, orthogonality remains the better glue. If the second fails, persistence is an elaborate restatement of a distance penalty. Both outcomes would be worth reporting.