Scale-Free Bargaining Between a Robot's Arm and Legs

Legged robots with arms are often controlled by two separate modules. One is a planner that moves the arm precisely. The other is a learned walking policy that keeps the body balanced. Splitting control this way makes each part easier to build, transfer from simulation to hardware, and swap out. The cost is coordination. Today the arm usually dictates and the legs absorb: the arm plans a motion, predicts the forces it will push onto the body, and the walking policy braces against them. The legs never get to say "that reach will tip me over, lean less," and the arm never learns that a small crouch would have made its reach easy. The obvious fix is to add a cost for each module and minimize a weighted sum. That brings back an old problem from social choice: the two costs have no common unit. Whichever module reports larger numbers wins. The weights must then be retuned every time someone rescales a reward, retrains a policy, or bolts on a different arm.

We propose to treat the arm and the base as two agents bargaining over a small shared variable: the body pose offset (height, pitch, roll) and arm motion timing. They bargain with a method that never compares their cost magnitudes. Each module reports only the direction in which it would like the shared variable to move, plus where its ideal value lies. A mediator combines these unit-length directions, weighting each by how far that module currently is from its ideal, measured in the shared physical coordinates. This is the Direction-based Bargaining Solution, which was built for exactly the setting where utilities cannot be compared. The claim is testable. A decoupled controller that negotiates this way should recover most of the reachable workspace of a single end-to-end whole-body policy. It should keep the modularity of decoupled control. And its behavior should be unchanged when either module's cost is rescaled or even nonlinearly distorted, a property no weighted-sum scheme can have.

Where This Sits

Three strands of decoupled control motivate the problem.

Combining Learning-based Locomotion Policy with Model-based Manipulation splits a quadruped manipulator into a model predictive controller for the arm (MPC, a planner that re-solves a short-horizon optimization at every step) and a reinforcement-learned walking policy. The arm planner forecasts 0.8 s of reaction forces and torques, and the walking policy takes that forecast as input. The authors state that the base "treats the arm's movements as an inevitable disturbance." We keep their forecast channel and add the missing return path.

Versatile Loco-Manipulation through Flexible Interlimb Coordination blends a model-based limb controller with a learned policy through a binary per-limb mask. That arbitration is all-or-nothing. We need graded compromise over a shared variable.

Deep Whole-Body Control supplies the counter-argument. Its unified policy reaches 0.82 m³ of arm workspace against 0.58 m³ for a decoupled baseline. It avoids one objective swamping the other through "Advantage Mixing," a hand-scheduled blend of separate locomotion and manipulation learning signals. We take this paper as the target to match. Its reliance on tuned blending weights is precisely the scale sensitivity we aim to remove.

The method comes from interpersonal utility comparison. Cooperative Bargaining Games Without Utilities introduces direction oracles and proves two things under strongly convex hidden costs: convergence to Pareto-stationary points, and immunity to strictly increasing distortions of any agent's utility. It was tested only on spatial formation games and portfolio allocation, never on physically coupled controllers.

Two other results constrain the design. Policy Aggregation shows that utilitarian summation under heterogeneous scales is badly inequitable: Gini index 0.4392 versus 0.0188 for its quantile rule. Its occupancy-polytope volume computations, however, do not scale to continuous whole-body control, so we do not use them. On the Foundations of Expected Expected Utility shows that anchoring utilities to shared best and worst outcomes still leaves intermediate preference strengths incomparable. That result undercuts min-max normalization as a fix.

No paper in the file applies scale-free bargaining to a single robot's subsystems. The two multi-robot loco-manipulation works both use centralized scalar rewards: Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushing and Collaborative Loco-Manipulation with Dynamic Reward Curriculum.

Core Claim

Let $z$ be the negotiated variable. It is a low-dimensional body command: a height, pitch and roll offset plus a time-scaling factor on the arm trajectory.

The arm planner can compute how its tracking cost changes with $z$ by differentiating its optimization. It reports a unit vector $d_A$ that points downhill, and an ideal point $z_A^*$.

The walking policy is trained to accept $z$ as a command. Its learned value estimate (a prediction of future stability and survival) is differentiable in $z$. It reports its own unit direction $d_B$ and ideal $z_B^*$, typically the nominal stance.

The mediator updates

$$z \leftarrow z + \eta \sum_{i \in \{A,B\}} \lVert z - z_i^* \rVert \, d_i ,$$

where $\eta$ is a step size. Each module's pull is weighted by its physical distance from what it wants, never by the size of its cost.

Rescaling either module's cost by a constant, or squaring it, changes neither $d_i$ nor $z_i^*$. The negotiated outcome is therefore unchanged by construction. A weighted sum lacks this property.

One scale choice remains, and it should be stated plainly: the metric on $z$ (meters versus radians). It is a single, physically interpretable choice. It is shared symmetrically by both modules and fixed once per robot. It is not a per-reward tuning knob.

Research Program

The work has three parts.

First, we build the two oracles. The arm direction comes from MPC sensitivities with respect to the commanded body pose. The base direction comes from gradients of a critic conditioned on $z$. As a fallback, the base can instead answer pairwise comparisons, the comparison-oracle variant analyzed in the bargaining paper.

Second, we run the mediator at the arm planner's replanning rate, a few iterations per cycle and warm-started. The forecast channel of Ma et al. stays in place, so the base still anticipates forces.

Third, we confront non-convexity directly. The convergence proof requires strongly convex costs. Contact switching and joint limits violate this. We will measure how often the mediator oscillates or stalls, and whether restricting $z$ to a few dimensions keeps the costs locally well-behaved. We will report the failure regimes rather than claim guarantees we do not have.

The platform is simulated in Isaac Gym, matching the Go1 plus WidowX configuration of Fu et al. so their workspace numbers are comparable. The ALMA-style reach and push tasks follow Ma et al. Hardware trials on a Go1-class platform are a stretch goal.

Baselines

The naive reference is a decoupled controller in which the base holds a nominal stance and simply resists arm forces. Fu et al. report 0.58 m³ of workspace for their decoupled baseline. Our reimplementation should land near that number, which also serves as a sanity check.

The strongest decoupled competitor is the unidirectional forecast architecture of Ma et al. It reports a 57% reduction in end-effector deviation versus a locomotion-only base and tolerates 150 N of lateral force without stepping. We will reimplement it on the same robot, with the same walking-policy training budget and the same arm planner, and add a $z$ input that is fixed rather than negotiated. Beating it means larger reachable workspace and fewer tip-overs on hard reaches, with no loss in tracking error or push tolerance.

The strongest overall competitor is the unified policy with Advantage Mixing of Fu et al., which reports 0.82 m³ of workspace and 87% perturbation survival. We retrain it with its published curriculum and equal simulation steps. Matching roughly its workspace while keeping arm-swap modularity would count as success. Exceeding it is not required.

The most dangerous baseline is a weighted-sum mediator: the same two modules and the same shared variable, but the update uses raw, unnormalized gradients with a scalar weight chosen by grid search. It gets the same number of tuning trials we spend choosing $\eta$ and the metric on $z$.

We also run a strict-priority baseline, which keeps the base at nominal unless the arm is infeasible. Finally, two ablations isolate the key ingredients. One uses normalized directions without distance weighting. The other uses distance weighting on raw gradients.

Why not just…?

Why not just tune the weight on a weighted sum? At the nominal reward scale, a well-tuned weighted sum may perform as well as our method. We expect this and will report it. The advantage we claim is elsewhere: the tuned weight is tied to the current reward scales. The decisive test multiplies the base reward by 10, then squares it, then swaps in a retrained walking policy, each time without retuning. The weighted sum should drift toward whichever module became louder. The bargaining outcome should not move. This mirrors the skew that the bargaining paper observed for Nash and Kalai-Smorodinsky solutions under squared utilities. If the weighted sum re-tuned per condition matches us, our contribution narrows to eliminating retuning. That is still useful for modular stacks, but it is a smaller claim, and we will state it as such.

Why not just normalize each module's cost to [0, 1]? Boutilier's result says shared endpoints do not make intermediate preference strengths comparable. A min-max-normalized sum therefore still encodes an arbitrary exchange rate between arm error and stability margin. We include normalized-sum as a variant of the weighted-sum baseline under the same rescaling tests. If it proves robust in practice, that is a real finding against our need for bargaining.

Why not just put a stability cost inside the arm's MPC? This is a weighted sum inside one optimizer. It inherits the same scale problem. It also forces the planner to model the legged dynamics that the learned policy was introduced to handle. We test it as a single configuration with a tuned weight, under the same rescaling protocol.

Why not just prioritize balance and project the arm into the remaining freedom? Strict priority never lets the base give up margin to help the arm. That is plausibly why decoupled workspace sits near 0.58 m³. The strict-priority baseline measures this directly.

Proposed Experiments

We report four measures on every method: reachable workspace volume, end-effector tracking error, tip-over rate on reach tasks near stability limits, and maximum lateral push tolerated.

The decisive check is a scale-robustness curve. For each method, we plot how much the negotiated body pose and the task metrics change under affine and squared transformations of each module's cost, relative to the nominal condition.

Beyond that curve, we report two further results. The first is arm swapping: the base oracle and policy stay fixed while the arm planner is replaced by one with a different reach model. The second is a convergence audit of the mediator across contact changes: the fraction of cycles that settle, oscillate, or hit joint limits.

A multi-robot extension is deferred until the single-robot claim holds. In that extension, two quadrupeds carrying a shared payload negotiate the payload pose, compared against the dynamic-reward-curriculum setting of An et al. with its 90.3% dual-robot success.