A robot exploring a building sees only a small bubble of space around it. Everything past the edge of that bubble is unknown, and most planners simply assume it is empty floor, so the robot drives into dead ends and has to back up. Recent work trains neural networks to "hallucinate", meaning predict, the unseen walls and rooms from the visible part of a top-down map. These predictors reuse tools built for photo inpainting, the task of filling a hole in an image. That task is geometrically backwards from the robot's. A photo hole is surrounded by known pixels and can be filled from its rim inward. A robot's unknown region surrounds the known bubble and has to be predicted outward, with less context the farther out it goes.
We propose a map-completion network built for this outward setting. It grows its prediction ring by ring outward from what the robot has observed. Each cell carries a soft "how much do we trust this" score that shrinks with distance and with the network's own uncertainty. That trust score decides how much each cell may influence others through a cheap long-range attention that looks only along the cell's row and column. Indoor maps are dominated by straight, axis-aligned walls and corridors, so row-and-column attention is a natural way to carry a visible wall segment across a large unseen gap. Gating that attention by trust stops the network's context from being swamped by the 70–90% of the map that is still blank. The core claim can be tested directly. A predictor that expands outward with trust-gated attention should keep wall continuity and free-space connectivity accurate farther from the observed frontier than one-shot predictors of equal size. It should also give a usable anytime prediction, nearest rings first, at a fraction of diffusion latency.
Core Claim
Two failure modes from the literature compound in exploration maps, and we target them together. Local convolutions cannot connect a wall seen on one side of a room to its continuation on the other. ForesightNav's U-Net variant produces fragmented corridors (ForesightNav: Learning Scene Imagination for Efficient Exploration). Global attention fixes reach but attends mostly to empty placeholders when the map is largely unknown. The research file names this as an open problem for ViT-based completion such as Imagine Before Go.
Our model handles both with one quantity: a per-cell trust score $v_j \in [0,1]$. Observed cells start at $v_j=1$ and unknown cells at $0$. At recurrence step $k$, the network predicts the ring of cells adjacent to the current trusted region. Each new cell receives trust $v_j = \gamma \cdot c_j$, where $c_j$ is one minus the normalized entropy of its predicted occupancy distribution and $\gamma<1$ is a learned per-step decay. Long-range mixing uses criss-cross attention with the trust score folded into the weights:
$$a_{ij} = \frac{v_j \exp(q_i^\top k_j/\sqrt{d})}{\sum_{j' \in \mathcal{R}(i)} v_{j'} \exp(q_i^\top k_{j'}/\sqrt{d})}, \quad j \in \mathcal{R}(i).$$
Here $q_i, k_j$ are the query and key vectors of cells $i$ and $j$, $d$ is their dimension, and $\mathcal{R}(i)$ is the set of cells in the same row or column as $i$. Two passes give every cell a path to every other cell, as in CCNet. Earlier predictions keep $v_j<1$, so later steps can still revise them rather than freezing them. This targets the error accumulation reported for onion-peel and recurrent inpainting. Recurrence stops adaptively when the front reaches the planner's horizon or when the mean confidence of the newest ring drops below a threshold $\tau$. A long hallway therefore gets many steps and a cramped room few, which replaces the fixed recurrence counts criticized in Recurrent Feature Reasoning for Image Inpainting.
Relation to Prior Work
Three papers supply the method. Partial convolutions condition each convolution only on valid pixels and grow a binary valid mask layer by layer. We reuse the renormalization but replace the binary mask with continuous trust, and we run the growth outward rather than inward. That paper itself reports failure when holes lack surrounding context, which is exactly the outward case. RFR-Net runs partial-convolution peeling recurrently in feature space with dense attention shared across steps. We keep the recurrence but swap its quadratic attention for gated criss-cross attention, and we move from RGB photos to categorical occupancy grids. CCNet supplies row-and-column attention, reported at 11x less memory and 85% fewer FLOPs than non-local blocks. CCNet was built for fully observed images and has no notion of missing data. The trust gate is our addition.
Several navigation systems motivate the work by showing that hallucination helps and that current predictors are one-shot. CogniPlan samples several GAN layouts in a single pass. Imagine Before Go uses a masked-autoencoder ViT. ForesightNav compares U-Net and ViT imagination modules and notes that the ViT becomes expensive in large buildings. None of them expands from the frontier, and none gates attention by validity.
Two diffusion papers constrain the design. Both SePaint and PLMD refine iteratively, across noise levels rather than across space, and both pay for it in latency. PLMD runs only every 50 steps and still takes 10–20% of inference time. Our refinement is spatial and anytime, so the planner can consume the near rings before the far ones exist.
ProxMaP is the one result that partly cuts against us. It found that predicting only the robot's immediate vicinity avoids overfitting to global layouts. We treat that as a hypothesis to test rather than ignore. If trust-gated expansion does not beat a proximal-only predictor at the near rings, while also extending usefully beyond them, the long-range component is not earning its cost.
Research Program
We first build a training distribution that matches the outward regime. Random patch masks such as MAE's 75% produce scattered holes. Instead, we replay exploration trajectories in HM3D and MP3D scenes, accumulate the egocentric observed region at each step, and use the full top-down floor map as the target. The model predicts three classes per cell: free, occupied, and outside-building. The third class follows ForesightNav's interior mask and keeps the model from extending walls into open exterior space. Training unrolls the recurrence on the model's own predictions, with a per-ring cross-entropy loss and a calibration term that makes $c_j$ track actual correctness. Calibration is required because the trust gate is only meaningful if confidence is honest.
Second, we analyze where errors arise. Every map metric is reported as a function of distance from the observed frontier, in rings. We also report two structural measures. Wall continuity is the fraction of ground-truth wall segments crossing the unknown region that are predicted as a single connected segment. Connectivity error is the relative difference between shortest-path lengths through the predicted map and through the true map, for random point pairs. The second measure is what a planner actually consumes.
Third, we plug the predictor into fixed downstream planners without changing them: the PLMD-style goal-selection pipeline for ObjectNav on HM3D, and frontier exploration for PointNav in the ForesightNav setting. Any gain is then attributable to the map.
Baselines
The naive reference is the optimistic planner that treats unknown cells as free. For map metrics we also use nearest-neighbor fill. SePaint reports nearest-neighbor at 25.21 mIoU on nuScenes BEV coverage and 7.86 under extreme sparsity, so we expect it to be weak far from the frontier. It is nevertheless a hard bar at ring one. Beating it means higher accuracy at every ring beyond the first, and fewer replans in navigation.
The strongest one-shot competitor is a masked-autoencoder ViT completion module in the style of Imagine Before Go, retrained on our trajectory-derived masks. On ObjectNav it reports 37.7% SR on MP3D and 78.0% SR on Gibson. A second one-shot baseline is a U-Net of matched parameter count with the same three-class head, standing in for ForesightNav's and ProxMaP's backbones.
The strongest iterative competitor is diffusion. On navigation we compare against PLMD, which reports 65.6% success rate on HM3D v0.1. We run its released pipeline with our predictor swapped in at the same call frequency, and again at every step, to expose the latency trade-off. On categorical map quality, SePaint is the reference: 37.21 mIoU on nuScenes BEV coverage and 12.98 under extreme sparsity. We train our model on the same nuScenes split as a transfer check. For indoor occupancy-completion accuracy stratified by distance there is no reported result found on Lacuna, so all indoor map-quality baselines are retrained by us.
Fairness rules apply throughout. Every learned baseline uses the same training masks, the same three-class labels, a parameter count within 10% of ours, the same GPU-hour budget, and the same hyperparameter-search budget (an equal number of trials). Wall-clock latency is measured on one GPU at the same map resolution.
Beating the field requires three things together. We must match the ViT at rings one to three. We must beat it on connectivity error beyond ring five. And we must stay within PLMD's success rate while running at least five times faster per call.
Three ablations isolate the ingredients. The first removes the trust gate and uses plain criss-cross attention. The second uses binary rather than soft trust, so predictions freeze once made. The third removes the outward recurrence, giving one gated criss-cross pass over observed cells only.
Why not just…?
Why not just mask out unknown tokens in a ViT's attention and predict everything in one pass? This is the most serious threat. It directly removes placeholder dilution and costs nothing to implement. Our answer is mechanistic but unproven. With observed-only keys, every far cell attends to the same observed set and receives similar context. In outward expansion, cells near the far edge attend to intermediate predictions that already encode which way a corridor turned. Ablation three, a single gated pass with no recurrence, is exactly this alternative in our architecture. We also run it as a masked-key ViT. If either matches the full model beyond ring five, the recurrence is unnecessary and we will report a simpler gated one-shot model.
Why not just enlarge the convolutional receptive field, with dilation or global frequency-domain filtering as in anisotropic spectral gating? Frequency-domain filters do reach globally. However, they mix blank and observed cells indiscriminately unless masked, and they carry no notion of which cells are trustworthy. We include a matched FFT-mixing U-Net baseline. If it equals gated criss-cross attention on wall continuity, that would show the axis-aligned inductive bias is not what matters.
Why not just run a fast diffusion model with few denoising steps? SePaint and PLMD show that diffusion handles categorical maps and multimodality well. We do not claim to beat diffusion on raw mIoU. We claim a better accuracy-latency curve and anytime output. We sweep PLMD's step count down to the latency of our model and compare success rate at matched compute. If few-step diffusion dominates that curve, our contribution reduces to the analysis tools: the ring-stratified metrics and the connectivity metric.
Decisive Checks
The central experiment plots per-ring connectivity error for the full model, the masked-key ViT, the matched U-Net, and the three ablations on held-out HM3D and MP3D scenes. It is repeated on KTH floor plans to test transfer to real architecture. The hypothesis survives only if the full model's error grows more slowly with distance than every one-shot baseline, and the no-gate ablation degrades most when observed coverage falls below 20%.
Navigation outcomes must agree with map quality. We measure ObjectNav success rate and SPL on HM3D through the PLMD pipeline, and PointNav exploration speedup against the 1.5x that ForesightNav reports. Both must move in the same direction as connectivity error. Better maps that do not shorten paths would indicate the metric is wrong.
Finally, an anytime test truncates recurrence at each ring. It checks whether planning on partial predictions already recovers most of the travel-length gain, which would justify calling the predictor every step instead of every 50.
References
- ForesightNav: Learning Scene Imagination for Efficient Exploration
- Imagine Before Go: Self-Supervised Generative Map for Object Goal Navigation
- Recurrent Feature Reasoning for Image Inpainting
- Image Inpainting for Irregular Holes Using Partial Convolutions
- CCNet: Criss-Cross Attention for Semantic Segmentation
- CogniPlan: Uncertainty-Guided Path Planning with Conditional Generative Layout Prediction
- SePaint: Semantic Map Inpainting via Multinomial Diffusion
- Plug-and-Play Label Map Diffusion for Universal Goal-Oriented Navigation
- ProxMaP: Proximal Occupancy Map Prediction for Efficient Indoor Robot Navigation
- Butterworth as Attention: Anisotropic Spectral Gating for Pansharpening