A robot moving through a crowd has to decide two different things. The first is how cautious to be: how much personal space to give people, and how much travel time it will give up for that. The second is which way to go: pass the group on the left, pass on the right, or fall in behind it. Current systems mix these two choices together. Social navigation planners either run one fixed personality or switch between a few hand-built behaviors ("halt", "follow", "pass"). Methods that learn many alternative solutions to one task keep a small discrete library, and none of them offers a continuous caution setting. We propose a single policy that takes two separate inputs. One is a continuous safety budget that sweeps smoothly from assertive to deferential. The other is a discrete index over alternative routes, each of which is near-optimal at that budget. When the preferred route is blocked, the robot can switch to another route at the same level of caution. Today it would instead become more cautious until it stops moving, the well-known "freezing robot" problem.
The core bet is structural. Caution is a continuous quantity, so blending along it should produce valid behavior. Route choice is topological: halfway between "left" and "right" is a collision. A latent-conditioned policy, meaning one network whose behavior changes with an extra input code, should therefore give the two factors different geometry. Gradients from a single conditioned critic can also be decomposed exactly by reward component. This avoids the need for one critic per style that earlier multi-style work required.
Core Claim
The claim is that factorizing the policy as $\pi(a \mid s, c, z)$ gives social navigation something that neither a caution-conditioned policy alone nor a discrete diversity library provides: graceful fallback when the preferred path closes. Here $s$ is the robot's observation, $c \in [0,1]$ is a safety budget (the allowed fraction of time spent inside a person's 1 m comfort zone), and $z \in \{1,\dots,K\}$ indexes alternative routes. Concretely, at a fixed budget $c$, switching $z$ should recover success in obstructed scenarios at lower cost in time and jerk than lowering $c$. Interpolating in $c$ should trace a Pareto front of efficiency against personal-space compliance about as good as training a separate policy per budget. The Pareto front here is the set of settings where one metric cannot improve without the other getting worse. A jointly learned continuous latent that merges both factors should instead produce collisions when interpolated between route modes.
Relation to Prior Work
Three lines of work supply the redundancy side. SMERL trains one policy on a discrete latent. It maximizes mutual information between states and $z$, subject to staying within $\epsilon$ of optimal return, and shows that the best skill shifts as test perturbations grow. DOMiNO and DSP pose diversity as a constrained optimization over occupancy measures with a near-optimality floor. DOMiNO reports 1.5× to 2.0× higher reward than a single policy under severe perturbation on the DeepMind Control Suite. BMD preserves up to 10 discrete modes while fine-tuning diffusion policies. All four define near-optimality against one fixed objective. We reuse the SMERL-style constraint and discriminator, but make the optimality reference depend on the budget $c$. The redundancy set is therefore "near-optimal alternatives at this level of caution," which none of them define.
Two papers motivate the continuous caution axis. MultiCriticAL shows that a shared actor with a single critic collapses conflicting styles into "mushy" behavior, and that one critic per style fixes this. Its actor then interpolates between styles at test time. We keep its diagnosis but change the remedy. We split critics by reward component (task progress and personal-space cost), not by style. The number of critics then stays at two no matter how many caution levels exist. The Option Keyboard combines skills linearly through successor features. We rely on the same linearity, but only where it holds exactly: the Lagrangian of a fixed policy is linear in the cost weight. We do not assume skills compose linearly.
DOMiNO also warns against the "scalarization fallacy": a fixed weighted sum cannot reach non-convex parts of a Pareto front. We therefore condition on a constraint budget $c$, not on a reward weight. The per-budget multiplier is learned by dual ascent. The design is shaped by this warning, not in conflict with it.
On the application side, Falcon adds forecasting of pedestrian trajectories and a penalty for planned paths that cross those forecasts. It reaches 55.15% success with about 90% personal-space compliance on Social-HM3D, but runs a single unimodal policy. Social latent-space navigation has a controller choose latents of a pretrained human-motion VAE. It achieves average jerk of 0.13–0.15 against 0.58 for the CRI-Linear baseline. Its latent represents typical human motion, not redundant task solutions, and it has no caution setting. SOCIALGYM and Arena 3.0 implement caution as discrete sub-policy switching or heuristic "personality" parameters. They document the costs: generalization drops, and fast planners spending 5.6 s in private zones. Latent-action planning for legged robots shows continuous latent interpolation helps recovery from damage, but over a fixed, hand-provided expert library. Our library is discovered.
Method
The policy and critic take $(s, c, z)$. Training samples a budget $c$ per episode and a route index $z$ per episode. The objective for each $(c,z)$ is:
$$\max_\pi \; J_r(\pi_{c,z}) + \beta\, I(\tau; z \mid c) \quad \text{s.t.}\quad J_g(\pi_{c,z}) \le c,\;\; J_r(\pi_{c,z}) \ge \max_{z'} J_r(\pi_{c,z'}) - \epsilon .$$
In this objective, $J_r$ is expected task return (goal progress minus time), and $J_g$ is the expected fraction of time spent violating personal space. $I(\tau; z \mid c)$ is the mutual information between trajectory $\tau$ and route index, estimated with a discriminator $q(z \mid \tau, c)$ as in SMERL. The coefficient $\beta$ weights diversity, and $\epsilon$ is the allowed return gap.
The diversity term is switched on only for $(c,z)$ pairs that satisfy the near-optimality constraint, following SMERL's gating. The discriminator reads robot-trajectory features relative to nearby pedestrians. These are passing side, lateral offset, and time-to-closest-approach. It does not read raw pixels, which keeps diversity tied to meaningful route differences. This is the hand-feature dependence noted for DOMiNO and DSP, accepted here deliberately.
The critic has two heads, $Q_r(s,a,c,z)$ and $Q_g(s,a,c,z)$. The actor maximizes $Q_r - \lambda(c)\,Q_g$, where the multiplier $\lambda(c)$ is a small monotone network updated by dual ascent on the constraint violation at each sampled $c$. For a fixed policy this combination is exact. Style interference therefore reduces to whether the two heads generalize across $c$, which is a much narrower question than per-style critics.
At deployment, a short-horizon selector sets $z$. It scores each $z$ at the operator's budget by combining $Q_r - \lambda(c) Q_g$ with a Falcon-style forecast of pedestrian positions. It switches $z$ when another index clearly dominates, with hysteresis to prevent rapid back-and-forth switching. Lowering $c$ is allowed only if every $z$ is predicted to violate the budget.
Research Program
The program asks three questions in order. First, does one budget-conditioned network trace a Pareto front comparable to separately trained policies? Second, does the discrete $z$ axis produce real route alternatives at each $c$? Third, do those alternatives, rather than state-conditioning alone, drive recovery when the preferred route is obstructed? Development runs in Arena 3.0's Flatland and the CRI-Linear circle-crossing setup, where sweeps are cheap. The final method is then trained on Social-HM3D with Falcon's forecasting module as the observation backbone, so it is directly comparable to Falcon's reported numbers. SocNavBench's 33 replayed episodes serve as a held-out, non-reactive stress test.
Baselines
The naive reference point is Falcon run at its published configuration, a single policy with a fixed caution level. Falcon reports 55.15% success and about 90% personal-space compliance on Social-HM3D. We retrain it with our seeds and training steps. To beat it, our method at a matching budget must show higher success at equal or better compliance, or equal success with lower freezing time.
The strongest competitor is a budget-conditioned policy without the route index: our architecture with $K=1$. This is the natural "just condition on the trade-off" answer. It already gives a continuous dial and could, in principle, pick a different side from the state alone. It gets identical data, network size, training steps, and tuning budget for $\beta$-free hyperparameters. No published number on these benchmarks exists for it, since no reported result was found on Lacuna for a caution-conditioned social navigation policy. Beating it means significantly higher success and lower freezing time in the obstructed-route scenarios described below, with no loss of Pareto hypervolume in unobstructed ones.
The Pareto reference is a set of separately trained Falcon-style policies at five fixed budgets, given five times the compute. We do not need to beat it. Matching its front within seed variance shows that sharing one network costs nothing.
The redundancy reference is SMERL with discrete $z$ and no budget input, trained at a single mid-range budget.
The switching reference is the discrete Multipolicy Decision Making used in SOCIALGYM, with Halt, GoAlone, Follow, and Pass sub-policies trained with PPO. SOCIALGYM reports a generalization gap for this setup but no exact numbers. On SocNavBench, we compare against the published extremes: Social Forces at 32/33 success with long paths, and SA-CADRL with 40 collisions. The shared targets are shorter paths than Social Forces and fewer collisions than SA-CADRL.
Two ablations isolate the key ingredients. The first replaces the two-head critic with one scalar critic conditioned on $(c,z)$, testing the MultiCriticAL interference claim in the continuous setting. The second replaces the factorized $(c,z)$ with a jointly learned two-dimensional continuous latent trained with the same diversity objective. This tests whether the discrete/continuous split matters.
Why not just…?
Why not just condition on the trade-off weight and let the state pick the route? This is the real threat. A budget-conditioned policy sees the pedestrians, so it may already take the free side when one side closes, and the route index would then be redundant. The obstructed-route experiment settles this directly. If the $K=1$ baseline recovers as well as the full model, with equal success, freezing time, and jerk, the redundancy claim is false. The note would then reduce to the two-head critic result. Our reason to expect a gap is SMERL's observation that a single optimal policy commits to one strategy and degrades under perturbations it did not train on. It is an expectation, not a proof.
Why not just blend the outputs of a cautious and a fast policy? Averaging velocity commands from policies that pass on opposite sides points the robot at the obstacle. MultiCriticAL found that middle-ground behavior without a learned conditioning space was non-functional. We still run output blending of the two endpoint policies from the separate-policy set as a cheap sanity baseline. We expect it to do reasonably well along the caution axis within one route and badly across routes. That pattern itself supports the factorization.
Why not just keep a menu of discrete behaviors with a state machine? Discrete switching is how SOCIALGYM and Arena 3.0 handle personality, and it produces jumps at switching boundaries. The jerk gap reported for continuous latent control in social latent-space navigation, 0.13–0.15 versus 0.58, suggests continuity matters. Our method is also discrete along routes, so the fair comparison is specifically about the caution axis. We will measure jerk and thrashing near budget thresholds for both.
Why not just randomize the crowd heavily in training? Domain randomization makes one policy tolerate the perturbations it saw. It does not give that policy a second option when an unseen obstruction appears, and SMERL reports that randomized and adversarially trained policies fall short of a diverse repertoire under unmodeled shifts. We give the $K=1$ baseline the same randomization we use, so any remaining gap cannot be attributed to training-distribution breadth.
Proposed Experiments
The decisive experiment is an obstructed-route test. In Arena 3.0 and Social-HM3D episodes, we inject a slow pedestrian group mid-episode into the corridor the policy's current mode is using, at a fixed caution budget. The main metrics are success rate, freezing time (seconds at near-zero velocity), time to goal, average jerk, and personal-space compliance. We compare four responses: switching $z$, lowering $c$, the $K=1$ conditioned policy, and discrete switching. The prediction is that switching $z$ dominates lowering $c$ on freezing time and time to goal at equal compliance. The prediction fails if the $K=1$ policy matches it.
The second experiment is Pareto quality. We sweep $c$ over 11 values and compute hypervolume in the plane of time to goal and violation fraction. We compare against the five separately trained policies and the scalar-critic ablation. The prediction is parity with separate policies and a measurable loss for the scalar critic.
The third experiment is interpolation validity. For the factorized model, we interpolate along $c$. For the joint two-dimensional latent, we interpolate between codes the discriminator assigns to different passing sides. We then count collisions and constraint violations along each path. The prediction is near-zero added collisions along $c$ and a spike for the joint latent midway between routes.
Finally, we run the full method zero-shot on SocNavBench's 33 episodes. This checks whether it lands between the Social Forces and SA-CADRL extremes, on a benchmark where pedestrians do not react to the robot.
References
- One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL
- Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimality
- Discovering Diverse Nearly Optimal Policies with Successor Features
- Behavioral Mode Discovery for Fine-tuning Multimodal Generative Policies
- Multi-Critic Actor Learning: Teaching RL Policies to Act with Style
- The Option Keyboard: Combining Skills in Reinforcement Learning
- From Cognition to Precognition: A Future-Aware Framework for Social Navigation
- Exploring Social Motion Latent Space and Human Awareness for Effective Robot Navigation in Crowded Environments
- SocialGym: A Framework for Benchmarking Social Robot Navigation
- Demonstrating Arena 3.0: Advancing Social Navigation in Collaborative and Highly Dynamic Environments
- Planning in Learned Latent Action Spaces for Generalizable Legged Locomotion