A cheap robot usually senses the world with a planar LiDAR, a single laser that spins in one horizontal plane at one fixed height. Anything above or below that plane is invisible. A tabletop at 75 cm is missed by a beam at 20 cm, so the robot sees four chair-sized legs and drives into the edge of the table. Learned "hallucination" models try to infer the missing geometry from the scan alone. They work on the floors they were trained on and break on new layouts: they smooth over doorways and misread isolated objects. This proposal argues the bottleneck is training data, and that the missing data can be produced cheaply and reliably by simulation. Two pieces of physics carry the load. The first is gravity: every overhang must rest on something, and that support usually shows up in the scan plane as legs, brackets or cabinet sides. The second is the physics of the laser itself: a beam is a cone of finite width, not an ideal line, so it returns mixed ranges at edges and drops out on glass or shiny surfaces.
In two sentences, the idea is this. We procedurally generate millions of indoor scenes whose furniture is physically supported and stable, and render them through a physical model of a 2D laser rather than an ideal ray caster. We then train a laser-only model that maps a real planar scan to the true obstacle distance across the robot's full height. Because the input is a range scan rather than a camera image, the gap between simulation and reality reduces to a small set of measurable sensor effects. That is the setting where physics-based synthetic data has historically transferred without any real labels.
Core Claim
A planar scan contains information about off-plane geometry only through the physical regularities that tie the two together, and the key regularity is support under gravity. A model trained on scenes that respect support constraints learns to read "four small blobs in a rectangle" as "a surface above them," and "a frame with no return in between" as "a glass pane." A model trained on a few hundred real scenes memorizes layouts instead. We claim that a generator combining gravity-consistent procedural layouts with a beam-footprint laser model will produce a scan-to-full-height-distance predictor with two properties. Trained with zero real labels, it will match the real-data-trained model on in-distribution floors. It will also substantially outperform that model on held-out layout types, specifically perpendicular furniture, isolated objects in open halls, and narrow doorways, where real-data training is known to fail.
Positioning Against Prior Work
The motivating evidence comes from Hallucinating Robots. It trains a convolutional autoencoder on 27,904 real scan/depth-slice pairs from one floor of a building, using only Gaussian noise and horizontal flips as augmentation. Its authors report over-smoothing that closes doorways and degraded performance on unfamiliar arrangements. We keep its laser-only setting and its output representation. We replace its data source. A second group of papers lifts the scan using a camera. Parse Geometry from a Line extrudes the scan into a vertical "curtain" and learns the residual depth. Inductive fusion pools the laser points into a global context for depth completion. Both create their planar inputs by slicing one line out of real KITTI or NYU depth data, which is an idealized, zero-width beam. Neither generates training scenes. A third group abandons the laser altogether. The pseudo-laser systems of Vision-based irregular obstacle avoidance and Monocular camera-based complex obstacle avoidance take a column-wise minimum over monocular depth. They inherit lighting sensitivity and failures of the floor segmentation, and they flatten away vertical clearance. We treat these as the strongest camera-based competitor in navigation rather than as the same task.
On the synthetic-data side, three papers shape the design. ANYU shows that geometric diversity matters more than semantic realism: injecting random CAD shapes improves iBims-1 RMSE from 1.493 to 1.365. It also shows that returns diminish once synthetic data exceeds about half the training mix, which it attributes to unmodeled sensor noise and rendering. This is a constraint on our design, and it is the reason we stay in the range domain, where rendering is not needed. ANYU's shapes also float freely, and our ablation tests whether that matters when the input is a planar slice. The method template comes from 3D nuclei segmentation. It trains purely on synthetic ellipsoids passed through a physical point-spread function with photon noise and reaches 92.20 to 94.26% accuracy on real microscopy. Our beam-footprint laser model plays the role of that point-spread function. Finally, VioLA completes geometry at the laser height with Stable Diffusion. This raises its Redwood alignment success from 66.7% to 74.1%, but its authors warn that it hallucinates geometry that does not exist. That warning is our reason to put physics into the generator of training data rather than into an unconstrained image prior at test time.
Research Program
The generator has three layers.
The scene layer samples rooms and corridors, including doorways of varied widths, and places furniture as support assemblies rather than as loose shapes. Tables sit on legs, pedestals or trestles. Shelves and monitors hang on wall brackets. Countertops rest on cabinets. Stacked boxes are dropped in a rigid-body simulator and kept only if they settle at rest. Glass partitions and doors appear with frames, mullions and handles. Layout sampling deliberately covers arrangements underrepresented in real buildings: furniture perpendicular to walls, islands in open halls, and clutter near doorways.
The sensor layer replaces the ideal ray. Each beam is a cone with the divergence of a target device. Its return is computed from the intensity-weighted mixture of surfaces inside the cone, which produces the mixed-pixel ranges seen at table edges and door frames. Dropout probability depends on incidence angle and surface reflectance. Glass partly transmits and partly reflects according to angle. Mirror-like surfaces can create ghost returns. Sensor attitude is perturbed by pitch and roll drawn from a simple vehicle-dynamics model of braking and turning, because a forward pitch drives the beam into the floor. These parameters are fit to a short calibration recording of the real target sensor against known targets; no labeled scenes are used for this fit.
The label layer computes, for each bearing, the nearest obstacle distance inside the robot's swept height. This is the same 1D target as Hallucinating Robots. We additionally record a few height bands so that clearance under tables is preserved rather than discarded.
The learner is deliberately conventional. We use the Hallucinating Robots autoencoder with skip connections, its γ-scaling and its RMSLE loss, so that any gain can be attributed to data rather than architecture. A secondary experiment adds temporal context over a few consecutive scans.
Baselines
The naive reference is the raw 2D scan used directly as the obstacle distance. It encodes the planar blindness we aim to remove. In Gazebo scenes with irregular furniture, the cited papers report this reference at 15–21% navigation success (Vision-based irregular obstacle avoidance) and 23% in Scene 7 (Monocular camera-based complex obstacle avoidance).
The strongest laser-only competitor is Hallucinating Robots trained on real paired data with Gaussian noise. It is the only published method for the same input/output mapping. We will use the authors' Care-O-bot 4 protocol (train on the second floor, test on the third) if the data are obtainable. Otherwise we will collect an equivalent multi-floor dataset with a planar LiDAR and an RGB-D camera, using the same 1D projection labels. Both models get identical architecture, loss and hyperparameter search budget. The file reports no numeric test RMSLE for this method, and no reported result was found on Lacuna for its final test RMSLE, so we re-run it ourselves. Beating it means three things: equal or lower RMSLE on the in-distribution floor with zero real labels; clearly lower RMSLE and fewer closed doorways on held-out layout types; and higher recall on overhangs and glass at matched false-positive rate.
A second strong laser-only baseline is our own: ideal ray-casting through real scanned meshes from ScanNet and Redwood. These scenes are real geometry and therefore already obey gravity. Each baseline is given the same number of training pairs and the same compute.
The strongest camera-based competitor in closed-loop navigation is the pseudo-laser system, reported at 89% success in its Gazebo scenes. We will run it alongside our policy in the same Gazebo scenes and with the same DRL training recipe. Our goal there is to reach comparable success without a camera, or to beat it in low light and on reflective floors.
Two ablations isolate the key ingredients. One removes the support constraints, keeping the same object shapes but placing them ANYU-style at arbitrary heights. The other replaces the beam-footprint sensor model with ideal rays plus tuned Gaussian noise.
Why not just…?
Why not just slice real 3D scans instead of building a generator? This is the most serious threat. Scanned meshes are physically valid by construction. Our answer is not to argue against it but to test it directly. We have two reasons to expect it to fall short. First, depth-sensor meshes typically have holes where glass should be, so they cannot supply glass labels. Second, their layouts reproduce the same regularities of real buildings that Hallucinating Robots overfit to. If sliced scans plus our sensor model match the procedural generator on held-out layouts, we will report that the sensor model is the contribution and the layout generator is not. The combined dataset is also a planned condition.
Why not just add more noise to the existing real-data model? Hallucinating Robots already tuned noise (σ=0.02) and still failed on unfamiliar layouts. Noise does not create new co-occurrences between legs and surfaces. The ideal-ray-plus-noise ablation measures how much of the gain comes from sensor physics versus layout diversity.
Why not just add a camera and use a pseudo-laser? That changes the hardware, and it brings back the lighting and segmentation fragility the pseudo-laser papers themselves report. The Gazebo comparison shows where each wins. If the camera system dominates everywhere, laser-only hallucination is the wrong target. If ours matches it, the cheaper sensor suffices.
Why not just use random floating shapes, which ANYU found sufficient? ANYU's input was a full image, in which a floating shape is visible. In a planar scan, an unsupported overhang leaves no trace at all, so training on such shapes teaches the model that overhangs are unpredictable. The no-support ablation tests this. We predict overhang recall drops sharply while wall and doorway accuracy stays similar.
Proposed Experiments
The first decisive check is zero-real-label transfer. We train only on synthetic data and evaluate on the held-out real floor. Metrics are near-field-weighted RMSLE, a doorway-closure rate (the fraction of traversable gaps of at least the robot's width that are predicted as blocked), overhang recall, and glass recall. Each is stratified by layout type, so the perpendicular-furniture and open-hall cases are reported separately.
The second check is the ablation matrix: support constraints on or off, crossed with sensor model versus ideal rays plus noise. We also vary the ratio of synthetic to real data, to see whether ANYU's diminishing-return threshold appears in the range domain.
The third check runs in closed loop. A DRL navigation policy is trained once on each perception module's output and evaluated in Gazebo café, cabinet and shelf scenes, plus a small physical deployment with tables, wall shelves and a glass partition. We report success rate and collisions per kilometre.
The result that would falsify the proposal is the following: the gravity-consistent generator gives no gain on held-out layouts over real-data training, or the no-support ablation performs as well as the full generator. Either outcome would mean that physical support is not what makes off-plane geometry predictable from a single line.
References
- Hallucinating Robots: Inferring Obstacle Distances from Partial Laser Measurements
- Parse geometry from a line: Monocular depth estimation with partial laser observation
- Depth Completion via Inductive Fusion of Planar LIDAR and Monocular Camera
- A Vision-based Irregular Obstacle Avoidance Framework via Deep Reinforcement Learning
- Monocular Camera-Based Complex Obstacle Avoidance via Efficient Deep Reinforcement Learning
- Virtually Enriched NYU Depth V2 Dataset for Monocular Depth Estimation: Do We Need Artificial Augmentation?
- Nuclei Segmentation of Fluorescence Microscopy Images Using Three Dimensional Convolutional Neural Networks
- VioLA: Aligning Videos to 2D LiDAR Scans