Sidestepping to See: Vantage-Point Planning for Holonomic Robots in Occluded Crowds

A robot moving through a crowd is partly blind. People and pillars block its sensors, and the worst accidents happen when someone steps out from behind such a blocker straight into the robot's path. Most robots handle this badly in one of two ways. They either ignore what they cannot see, or they assume danger lurks behind every blocker and slow to a halt. There is a middle option: move a short distance sideways so the sensor can see around the blocker. This proposal asks whether a holonomic robot, one that can slide in any direction without turning first, can use small sideways moves to see behind blockers before it reaches them. The question is whether it can do this cheaply enough, and politely enough, to beat robots that only infer what is hidden or only brake.

The claim is that in occluded crowds, the real advantage of holonomic drive is translation, not rotation. The robot can buy a new line of sight for a few tens of centimetres of lateral motion, inside a footprint that does not intrude on nearby people. Rotating a camera toward a blind spot can be matched by a gimbal or a 360° sensor. Translating the sensor's optical centre sideways while still moving forward cannot be matched by a differential-drive base. A second consequence follows. Once a full-surround sensor handles coverage, the holonomic robot's heading no longer serves perception, so it can be used as a social signal pointing where the robot intends to go. We build a planner that chooses lateral offsets to shrink the hidden region ahead, subject to personal-space costs. We then test, in a factorial design, which part of any gain comes from translation, which from heading, and which from simply having better sensors.

What the literature already establishes, and what it does not

Three bodies of work each supply one piece.

On the occlusion side, Occlusion-Aware Crowd Navigation Using People as Sensors infers hidden obstacles from how visible pedestrians swerve, using a variational autoencoder over a bird's-eye occupancy grid. It reports 91% success against 79% for limited-sensor baselines and 97% for a full-visibility oracle. The method is entirely passive, runs on a differential-drive Turtlebot, and its authors name static occluders such as walls and corners as an open limitation. Safe Occlusion-Aware Autonomous Driving via Game-Theoretic Active Perception shows the active alternative for cars: nudging forward or sideways shrinks the "hidden set" of states a concealed agent could occupy, which permits faster safe trajectories. Its active moves are car manoeuvres against a single worst-case adversary, with no notion of personal space. Occlusion-Aware Risk Assessment for Autonomous Driving in Urban Environments seeds unseen road regions with randomised phantom agents and reduces 95th-percentile collision rate from 12.64% to 2.61%. Its particles depend on HD road maps, which indoor social spaces lack.

On the holonomic side, iRotate: Active Visual SLAM for Omnidirectional Robots decouples camera heading from path to improve mapping, with up to 39% shorter paths. It only rotates and only works in static scenes. Search-based Kinodynamic Motion Planning for Omnidirectional Quadruped Robots cuts a via-point course from 54 s to 32 s by strafing, but assumes full observability. TidyBot++ supplies an open powered-caster holonomic base on which a learned policy succeeded 90% versus 40% on a differential base for a manipulation task. That result motivates the hardware choice, but it says nothing about crowds.

On the social side, Frozone is the paper that most directly motivates this work. Because its camera is fixed to a differential chassis, it deliberately limits how far the robot may swerve so the tracked pedestrian stays in view. This is a perception constraint that becomes a motion constraint, and holonomic drive removes it. Incorporating Gaze into Social Navigation shows that a robot's virtual gaze communicates turning intent far better than LEDs (50% versus 0% zero-shot comprehension). Its 15% versus 48% conflict-rate result concerns human pedestrians whose gaze is incongruent with their motion, not robots. That makes it suggestive rather than conclusive about whether a robot sliding sideways while facing forward confuses people. We treat that as an open question to measure, not a premise.

Crowd-navigation reinforcement-learning methods, LM-SARL and MP-RGL, already output holonomic velocity commands but assume full visibility, and they are typically deployed on non-holonomic hardware. No paper in this collection combines social inference about hidden regions with active translational repositioning under personal-space costs. Our work differs from PaS by adding active vantage-point motion and static occluders. It differs from Zhang and Fisac by being probabilistic rather than worst-case, map-free, crowd-aware, and holonomic.

Core Claim

The planner keeps a belief over where hidden agents might be. The belief has two parts. The first is a People-as-Sensors-style occupancy inference for regions shadowed by people. The second is phantom particles seeded in unobserved free space next to static occluders, using the occupancy map rather than an HD map, with walking-speed kinematics. Each candidate short-horizon trajectory $\xi$ is scored by

$$J(\xi) = J_{\text{progress}}(\xi) + \alpha\, J_{\text{prox}}(\xi) + \lambda\, R_{\text{hidden}}(\xi) + \beta\, J_{\text{heading}}(\xi),$$

where the terms are defined as follows. $J_{\text{progress}}$ penalises time-to-goal. $J_{\text{prox}}$ is a learned or hand-set personal-space cost around tracked people, in the spirit of Robots learning how and where to approach people. $R_{\text{hidden}}$ is the probability mass of hidden agents that would first become visible inside the robot's stopping region, evaluated using the sensor visibility predicted along $\xi$. $J_{\text{heading}}$ penalises heading that disagrees with the intended travel direction.

The key property is that $R_{\text{hidden}}$ depends on where the sensor will be. A lateral offset therefore lowers risk by revealing the region earlier, not by slowing down. Holonomic kinematics make that offset cheap in $J_{\text{prox}}$ and $J_{\text{progress}}$, whereas a differential base pays for it with an arc. The prediction is specific: the gain should concentrate in scenarios with static or slow occluders close to the path, such as corridor corners, pillars, and stationary groups. It should vanish in open crowds where nothing is hidden.

Research Program

The work builds a single sampling-based planner, a model-predictive scheme over short trajectories with explicit visibility ray-casting. The same planner runs under three motion models: differential drive, differential drive with a pan gimbal, and holonomic. It also runs under two sensor models: a limited-field depth camera and a full 360° LiDAR. This keeps the comparison about kinematics and sensing, not about planner quality.

A learned variant replaces the hand-built scorer with a PPO policy, the reinforcement-learning algorithm Proximal Policy Optimization, trained on the PaS latent plus particle belief. This checks that conclusions are not an artifact of the planner. The analysis has three parts. The first decomposes success and near-miss gains into rotation-only, translation-only, and both. The second measures how far sideways the robot actually moves and how often it enters personal space. The third, through a small human study, measures whether facing the direction of intended travel while strafing reads as legible to pedestrians.

Baselines

The naive reference point is the same planner with $\lambda = 0$, which ignores hidden regions on a differential base with a limited-field camera. This reproduces the "standard limited-sensor" condition that Occlusion-Aware Crowd Navigation Using People as Sensors reports at 79% success.

The strongest existing method is PaS-VAE with its PPO planner, which reports 91% success and 13.91 s navigation time in its ORCA-based partially observable simulator. We run the released configuration where available, or a faithful reimplementation otherwise, in that simulator with identical crowd seeds. We also run it in our augmented scenarios with static occluders, giving it the same training-step budget and hyperparameter search as our learned variant. Beating it means higher success and lower late-detection near-misses on the static-occluder split, with no regression on the original split. The target is to close at least half the gap to the 97% oracle on the hardest split.

A conservative baseline treats every occluded cell as holding a worst-case walker, in the forward-reachable-set style criticised by Safe Occlusion-Aware Autonomous Driving via Game-Theoretic Active Perception. It bounds the freezing end of the trade-off.

Frozone is run on the differential base as the strongest anti-freezing social baseline. It reports more than 80% lower freezing rate and up to 50% higher success than prior methods on its hardware. Those numbers are not directly comparable to our setup, so we compare on our scenarios only.

On the open CrowdNav circle-crossing task, MP-RGL's reported 96% success and 2% collision rate in the invisible-robot setting, from Relational Graph Learning for Crowd Navigation, is a sanity floor that our planner must not fall below when there is nothing to occlude.

The ablation that isolates the key ingredient is the holonomic robot with the information term on but lateral offset forbidden, so it may rotate but must translate along its heading. This is paired with the converse, translation allowed but heading locked to velocity.

Why not just…?

Why not just put a 360° LiDAR on a differential-drive robot? This is the most serious threat. It removes every field-of-view limitation, and it is the condition under which rotation-based arguments for holonomic drive, iRotate-style, collapse. It does not remove line-of-sight shadows behind people and corners, which Occlusion-Aware Crowd Navigation Using People as Sensors identifies as the cause of jack-in-the-box failures. For that reason every holonomic-versus-differential comparison is run with the 360° sensor, so the only difference is whether the sensor can be translated sideways cheaply. If differential plus 360° LiDAR matches holonomic plus 360° LiDAR on the static-occluder split, the core claim is wrong and we will report it as such.

Why not just add a pan gimbal to a differential base? A gimbal reproduces the decoupled gaze of iRotate and lifts Frozone's need to bound swerving for tracking. It cannot shift the viewpoint. The differential-plus-gimbal condition, combined with the rotation-only holonomic ablation, measures exactly how much of the gain gimbals capture. Our prediction is most of the gain in open crowds and little at corners.

Why not just rely on People as Sensors? Passive inference needs informative bystanders. Its own authors exclude static occluders, and Behavior Planning of Autonomous Cars with Social Perception notes that social inference fails when agents are inattentive. We test this directly with a split where simulated bystanders are non-reactive to the hidden agent. If PaS-VAE alone stays near its 91% there, active vantage-point motion adds little, and the honest conclusion would be to use passive inference only.

Proposed Experiments

The simulation backbone is the ORCA-based partially observable simulator of PaS, extended with walls, pillars, and L-shaped corridor corners. It is complemented by HabiCrowd for photorealistic indoor scenes with real architectural occluders. HabiCrowd reports no baseline success or collisions-per-distance numbers on Lacuna. SocNavBench provides replayed ETH/UCY pedestrians, with the caveat that its non-reactive humans cause unavoidable collisions, so we report collision counts alongside its baselines (Social Forces 32/33 success; SA-CADRL 40 collisions). CrowdNav serves as the no-occlusion sanity check.

Metrics are success, collision, time-to-goal, freezing rate, and personal-space intrusion time. The headline metric is first-detection distance: how far from the robot a previously hidden agent is when it first becomes visible. This is the quantity active vantage-point motion should change and passive inference cannot.

Hardware trials use a TidyBot++ base against a differential base carrying the same LiDAR. They run at corridor corners and pillars with scripted confederates emerging from behind cover, plus a legibility study in which bystanders predict the robot's next direction under heading-forward versus heading-aligned-with-intent strafing.

The decisive check is a single interaction effect. On static-occluder scenarios with 360° sensing, the holonomic planner with lateral offsets should shift the first-detection distance distribution outward and raise success toward the oracle relative to both differential plus 360° LiDAR and the rotation-only ablation. On open-crowd scenarios that gap should be near zero. If the gap appears everywhere, the gain is generic manoeuvrability rather than vantage-point gain. If it appears nowhere, holonomic drive buys nothing for occlusion that a better sensor does not.