Teaching Legged Robot Fleets from Planners That Ignore Their Feet

The fastest way to coordinate thousands of warehouse robots is to train a small neural policy that runs on each robot and imitates a large search-based planner. That planner thinks of each robot as a dot moving between grid cells. For wheeled carts this is roughly true. For legged robots it is not. When a grid planner commands an abrupt stop, a sharp turn, or a dodge around a neighbor, a four-legged robot often cannot comply without its feet slipping on the floor. It then falls behind schedule, and the whole fleet's timing starts to break down. This proposal treats the planner as a flawed teacher. Its demonstrations are good at avoiding collisions and poor at respecting what a legged body can physically do. We learn, per robot and from local information only, how much to trust each demonstrated command.

Concretely, we train a small "executability critic." It is a network that looks at one robot's local surroundings and a candidate command, and predicts how badly the robot's own low-level walking controller will slip or drift when it tries to carry that command out. The critic is trained from cheap single-robot simulation. It then re-weights the imitation loss of a decentralized fleet policy, so that physically unexecutable teacher commands contribute little gradient. The goal is to keep the scalability of planner distillation, reduce foot slip and the timing cascades it causes, and avoid both expensive physics optimization per motion and a centralized contact-aware planner.

Core Claim

Three bodies of work each solve one piece of this problem and assume the other pieces away.

Scalable fleet imitation shows that decentralized policies can be distilled from search planners. Graph Neural Networks for Decentralized Multi-Robot Path Planning clones an optimal centralized planner into a message-passing policy. Deploying Ten Thousand Robots (SILLM) clones a bounded-suboptimal search solver and reaches 10,000 agents. Both treat agents as grid movers. SILLM itself notes that execution diverges from plans on real robots.

Work on foot sliding repairs physical infeasibility one motion at a time and at great cost. DynaRetarget optimizes each human motion into a feasible humanoid trajectory, at about a minute of compute per second of motion on a 112-core server. LAGOON trains a simulated tracking policy that projects a kinematic proposal onto a feasible quadruped gait.

Work on suboptimal demonstrations re-weights data by quality. Exponentially Weighted Imitation Learning for Batched Historical Data (MARWIL) weights actions by estimated reward advantage. MisoDICE filters mixed-quality multi-agent data with a jointly learned, linearly factorized value function. In both, quality means task reward, not physical executability.

Our claim is that, for legged fleets, the teacher's dominant flaw is local and physical: whether one robot's stance feet can hold the commanded motion. That makes the flaw estimable from a single robot's perspective without joint value learning. We define the weight for agent $i$ as

$$w_i = \exp\big(-\beta\, \hat{C}_\phi(o_i, a_i)\big), \qquad \mathcal{L}(\theta) = -\sum_i w_i \log \pi_\theta(a_i \mid o_i, m_i),$$

The symbols are as follows. $o_i$ is agent $i$'s local observation, including its body velocity and gait phase, nearby obstacles, and neighbor states. $a_i$ is the teacher's command. $\hat{C}_\phi$ is the executability critic's prediction of stance-foot slip plus tracking error. $\beta$ sets how sharply infeasible commands are discounted. $m_i$ is the message the policy receives from neighbors. $\pi_\theta$ is the decentralized fleet policy.

This is MARWIL's functional form with a different, locally computable signal. Because $\hat{C}_\phi$ is per-agent, the method does not need the joint value mixing whose convexity requirements MisoDICE had to engineer around. The price is that it cannot correct coordination-level suboptimality in the teacher, and we state that limit up front.

Research Program

The system has three trained parts and one fixed part.

The fixed part is the low-level controller. It is a PPO-trained quadruped locomotion policy in IsaacGym that tracks planar velocity and heading commands, trained in the style of LAGOON's tracking stage. It is trained once, before any fleet data exists.

The first trained part is the executability critic. We roll out the controller in single-robot simulation on sampled command sequences. These are drawn both from teacher plans and from random perturbations around them, with occasional static and moving obstacles standing in for neighbors. We record stance-foot horizontal velocity, falls, and deviation from the commanded pose over a short window. This is amortized single-agent compute: once trained, the critic costs one forward pass per agent per step. That is the direct contrast with DynaRetarget's per-motion optimization.

The second trained part is the teacher data. We run a SILLM-style search teacher on continuous-space versions of the lifelong warehouse maps. Grid moves are converted to timed waypoints, and each teacher command is labeled with its critic score.

The third trained part is the fleet policy. It is a SILLM-style communicating policy trained with the weighted loss above, and it keeps a collision-shield fallback for safety.

The main risk is coordination breakage. If agent $i$'s teacher command is down-weighted, its neighbors' demonstrated actions may have assumed $i$ would move as planned. We therefore study a second variant in which the critic score is also given as input to neighbors' messages, so that the policy can learn to yield to a robot that cannot turn quickly. Deadlock and collision rates are primary outcomes, not afterthoughts.

The analysis question is when a state-dependent feasibility signal matters. Our hypothesis is that slip depends on gait phase, current speed, and turn history, and that these interact with neighbor density. A fixed speed and turn-rate limit would then be either too conservative in open aisles or too permissive in crowds.

Baselines

The naive reference point is the decoupled stack. A SILLM-style policy trained with plain imitation on the unmodified teacher sends waypoints to the same low-level controller, and the controller does its best. This is what a practitioner would deploy tomorrow, so it is the bar every other variant must clear. For context, SILLM reports average throughput improvements of 137.7% over learning-based methods and 16.0% over search-based methods on large grid maps, and planning under 1 second per timestep for 10,000 agents (Deploying Ten Thousand Robots). No reported result found on Lacuna for any fleet planner executed by legged robots with physics, so we will produce that number ourselves.

The strongest existing competitor is an envelope-constrained teacher. The same search solver plans under hand-tuned speed, acceleration, and turn-rate limits fitted to the quadruped. Plain imitation is then applied to its output. This competitor attacks the same failure with far less machinery, which is why it is the most important comparison.

A reward-advantage baseline applies MARWIL with a critic trained on a fleet-level reward that includes a slip penalty. It tests whether a local executability signal beats a learned global value. MisoDICE is included where feasible as the principled multi-agent offline method, run on the same teacher data with a physical-slip-aware reward.

The key ablation replaces $\hat{C}_\phi$ with a state-independent function of command magnitude alone. This isolates whether the state dependence of the critic is what helps.

All variants share the teacher runs, the controller, the policy architecture, the training steps, and an equal hyperparameter search budget. The envelope baseline's limits are tuned on the same validation maps with the same number of trials that we spend tuning $\beta$.

Beating the envelope baseline means one of two things, on held-out maps at 100 to 1,000 agents. Either lower stance-foot slip and fall rate at equal or higher throughput, or higher throughput at matched slip. A result that merely trades throughput for slip along the envelope's own curve does not count.

Why not just…?

Why not just plan waypoints and let the walking controller cope? Evidence from single robots says tracking infeasible references fails often. DynaRetarget reports a 74.6% success rate for dynamically optimized references versus 37.9% for the prior method SPIDER on the OmniRetarget humanoid set (DynaRetarget). In fleets, execution delays propagate into plan violations (Concurrent Planning and Execution in Lifelong Multi-Agent Path Finding with Delay Probabilities). Still, this is an empirical question for quadrupeds on flat warehouse floors. The naive baseline will tell us whether the problem is large enough to matter. If slip there is rare, the proposal is answered negatively, and that is a useful result.

Why not just add kinematic limits to the planner? This is the real threat. If slip is determined mostly by speed and turn rate, a fitted envelope will match our method at a fraction of the complexity. The magnitude-only ablation and the envelope baseline test exactly this. We will also report slip as a function of gait phase and neighbor proximity at fixed command magnitude. If those curves are flat, we concede the envelope suffices.

Why not just filter demonstrations with a foot-contact cleanup and clone the rest? Kinematic zero-velocity cleanup fixes how motion looks, not whether forces balance, as the foot-sliding literature stresses. Hard-discarding also removes whole coordinated episodes, where soft weighting removes only the offending agent-steps. MisoDICE reports that cloning expert-only data on SMACv2 Protoss 5_vs_5 reached a 14.2% win rate versus 20.7% for its weighted method, which suggests filtering loses useful signal (MisoDICE). We include hard thresholding of $\hat{C}_\phi$ as a variant to check this in our setting.

Why not just run centralized contact-aware predictive control? Single-robot contact optimization already costs about a minute per second of motion (DynaRetarget). Distributed variants such as Distributed Data-Driven Predictive Control for Multi-Agent Collaborative Legged Locomotion rely on fixed communication topologies and linear models. We run a small-scale version (around 10 agents) only as an upper reference on slip.

Proposed Experiments

We will use the Warehouse, Sortation, and Random maps from the lifelong path-finding benchmark in Deploying Ten Thousand Robots, converted to continuous floors in IsaacGym with simulated quadrupeds. Fleet sizes run from 50 to 1,000. The upper end is limited by simulation cost, not by the policy.

The metrics are:

  • throughput (tasks completed per unit time);
  • mean stance-foot horizontal speed, which is the foot-sliding measure;
  • fall rate;
  • collisions;
  • deadlock rate;
  • plan-to-execution timing divergence;
  • per-step policy compute.

The first decisive check is whether the critic's predicted slip correlates with observed slip in multi-agent rollouts, even though it was trained on single-robot rollouts. If it does not, the local-feasibility premise fails.

The second decisive check is the envelope comparison on held-out maps and at fleet sizes larger than those seen in training.

The third decisive check is the neighbor-aware variant's deadlock rate relative to the per-agent-only variant. This tests whether down-weighting one robot breaks its neighbors' coordination.

A final transfer test changes floor friction without retraining the fleet policy, retraining only the critic, to see whether executability information can be updated independently of coordination.