TL;DR
Z-FLoc is a zero-shot floorplan localization framework that maps monocular camera sequences to 2D structural floorplans without requiring any environment-specific training. By extracting robust geometric primitives—lines and circles—from a bird's-eye-view reconstruction and matching them via a hybrid RANSAC loop, it achieves exceptional cross-environment generalization, drastically outperforming deep learning baselines on unseen buildings.
What Problem Does This Paper Address?
Indoor localization is crucial for applications like augmented reality and autonomous robotics. While matching camera images against pre-acquired 3D maps or image databases is highly accurate, building and maintaining these maps is labor-intensive. 2D floorplans are a highly attractive alternative because they are readily available for most commercial and residential buildings, compact, and completely invariant to changes in lighting or furniture.
However, localizing a camera within a 2D floorplan introduces a massive modality gap (perspective RGB images vs. top-down schematic drawings) and a coordinate frame gap (egocentric vs. allocentric views). Current state-of-the-art methods rely on deep learning to bridge these gaps, but they require extensive training data and fail to generalize to new, unseen environments without being retrained. Existing training-free alternatives exist, but they typically demand high-quality, high-coverage LiDAR point clouds or full $360^\circ$ panoramic images. Z-FLoc addresses these limitations by enabling zero-shot floorplan localization using only standard monocular image sequences.
Methods
Z-FLoc bypasses data-driven learning by focusing entirely on structural geometry. The pipeline consists of several distinct stages:
- Top-View Geometric Primitive Extraction: The system takes a sequence of RGB images and estimates a 3D point cloud using a visual foundation model (e.g., MapAnything). It segments wall and floor regions using Mask2Former to eliminate clutter like furniture.
- Wall Mask Filtering: To eliminate "ghost" artifacts from segmentation errors, the authors warp the images into a gravity-aligned frame. Because vertical walls exhibit consistent depth values per image column, they filter out any pixels that are closer to the camera than the dominant wall depth, retaining only the farthest structural wall.
- 2D Primitive Extraction: The filtered wall points are projected into a 2D Bird's-Eye-View (BEV) map. Line segments are extracted using seed-based region growing, while circular primitives (like pillars) are isolated using HDBSCAN clustering and fitted via RANSAC.
- Model Estimation via Hybrid Solvers: Instead of relying on a single geometric cue, Z-FLoc utilizes a hybrid RANSAC framework containing four distinct minimal solvers to determine the 2D similarity transform (scale, rotation, translation):
- Three-line (3L) solver: Uses three line correspondences.
- Two-line (2L) solver: Uses two line correspondences when an approximate metric scale is available.
- Two-circle (2C) solver: Uses two pairs of corresponding circle centers.
- Line-circle (LC) solver: Uses one line and one circle center.
- Model Scoring: Hypotheses are evaluated using a dual-component cost function:
- Consistency score ($C^c$): Rewards hypotheses where transformed BEV points align closely with floorplan walls.
- Violation score ($C^v$): Penalizes hypotheses that mistakenly place floorplan walls inside observed free space (negative space).
- Model Refinement: The top-ranked hypothesis is finally polished using non-linear optimization (Levenberg-Marquardt) with a Huber loss.
Datasets
The authors evaluated Z-FLoc on three distinct benchmarks:
- Gibson(t): A synthetic dataset containing 118 trajectories in smaller indoor environments ($<300\text{ m}^2$), captured with a wide field-of-view ($108^\circ$).
- LaMAR (HGE and CAB): A challenging, large-scale ($22,500\text{ m}^2$) real-world dataset captured across two university buildings. It features severe occlusions and a very narrow camera field-of-view ($48^\circ$). HGE contains both linear walls and cylindrical pillars, while CAB consists only of linear walls.
Key Insights and Ideas
- Geometry as the Ultimate Domain Bridge: Human-made environments are dominated by lines and circles. By reducing both the camera's 3D observation and the 2D floorplan to these fundamental geometric primitives, the severe modality gap between images and blueprints is entirely bypassed.
- The Power of Negative Space: Relying solely on wall alignment can lead to false positives, especially in repetitive architectural layouts (like long corridors). Introducing a violation score that penalizes layouts where a wall would block a known open pathway drastically improves robustness.
- Hybrid Solver Flexibility: Rather than committing to a single primitive type, the hybrid RANSAC approach dynamically adapts to the environment. If a room has no pillars, the line solvers dominate; if a room is defined by circular columns, the circle solvers take over.
Key Results
Z-FLoc demonstrates remarkable cross-environment generalization, using a single, fixed set of hyperparameters across all experiments.
- Cross-Scene Generalization (LaMAR CAB): When evaluated on the unseen CAB building (using models trained/tuned only on HGE), the deep learning baseline
F3Locfailed entirely ($0\%$ success rate).UnLocachieved a $50\%$ success rate at 100 frames but degraded sharply on shorter sequences. Z-FLoc achieved a 100% success rate at both 100 and 50 frames, and maintained $58.9\%$ accuracy even on ultra-short 15-frame sequences. - Synthetic Performance (Gibson(t)): Z-FLoc achieved a 99.5% success rate at 100 frames, outperforming zero-shot deep learning variants (which dropped to $8.0\%$ or lower) and even beating baselines that were explicitly trained on Gibson.
Limitations and Future Work
- Reconstruction Bottleneck: The pipeline's runtime is currently dominated by the dense 3D reconstruction step. While the localization components are fast, the system currently runs on the CPU and would need GPU acceleration or incremental processing for real-time deployment.
- Symmetric Ambiguities: In highly symmetrical environments (like identical, featureless hotel corridors), geometric matching can produce multiple high-scoring hypotheses. The authors suggest that integrating multi-modal filtering (tracking multiple pose hypotheses over time) could resolve this.
Summary updated