Designing Hardware for a Whole Front of Operating Modes

A robot, a reactor or a carbon-capture column is built once but run many ways. The same walking robot may be asked to move fast on one day and save battery on another. The same absorber may be pushed to high capture rates when carbon prices spike and run lean when energy is expensive. Work on "simultaneous optimization" (also called co-design: choosing the physical design and the operating policy in one optimization instead of design-first, control-second) has shown large gains over sequential pipelines. Yet almost all of it optimizes a single number, such as total annual cost, profit or one scalar reward. Methods that do handle several objectives let every point on the trade-off curve have its own hardware, which no one can build.

We propose to optimize the hardware for the best set of trade-offs its controller can reach after the hardware is frozen. The physical design becomes a "leader" whose score is how well a preference-conditioned controller (the "follower", one policy that takes a trade-off weight as input) can serve every preference the operator might later choose. In two sentences: sequential design fixes hardware for one objective, and per-point multi-objective co-design gives each trade-off its own hardware. We instead pick the single piece of hardware whose reachable trade-off curve is best, and we measure how much of the unattainable "one-design-per-preference" curve a single design can recover.

Core Claim

Existing co-design results support the claim that coupling design and operation beats separating them, but only for one scalar objective. Efficient Morphology-Control Co-Design via Stackelberg Proximal Policy Optimization treats body shape as a leader and the control policy as a follower. It differentiates through the follower's adaptation and reports an average 20.66% gain over BodyGen, but on a single locomotion reward. SoftZoo uses differentiable soft-body physics and reports a co-design score of 0.332 against 0.107 for control-only and 0.152 for design-only optimization, again with scalar task scores. The process-engineering results make the same point with costs. Joint optimization of rotating-packed-bed geometry and operating conditions cuts total cost by 9.4–12.7% (Simultaneous Optimization of Design and Operating Conditions for RPB-based CO2 Capture Process). Heat integration inside flowsheet optimization nearly doubles profit (Simultaneous optimization and heat integration of chemical processes). All of these papers motivate the proposal, and all of them assume the operator's objective is known and fixed at design time.

Our claim is that this assumption hides a second kind of coupling. The hardware determines not only the best achievable score but also the shape of the trade-off curve that operation alone can trace. Leg length or gear-like actuator scaling, for example, sets how steeply speed must be bought with energy. Hardware tuned for one weighting can leave the controller a narrow, poor front at other weightings. We predict that hardware chosen for flexibility will give up little at the preference a scalar co-designer targets and gain substantially elsewhere. We also predict the gain grows as the objectives conflict more strongly through the hardware.

Concretely, let $\theta$ be the hardware parameters and $\pi(a \mid s, \lambda)$ a controller conditioned on a preference vector $\lambda$ drawn from a distribution $p(\lambda)$ over the simplex. Let $J(\theta,\pi,\lambda) \in \mathbb{R}^m$ be the vector of $m$ returns (for example speed and negative energy) when that controller runs on that hardware. The leader solves

$$\max_\theta \; \mathbb{E}_{\lambda \sim p}\Big[\, \max_{\pi} \; \min_{i} \lambda_i \big(J_i(\theta,\pi,\lambda) - z_i\big) \Big],$$

where $z$ is a fixed reference point and the inner $\min_i$ is weighted-Chebyshev scalarization. We use Chebyshev scalarization because, unlike a weighted sum, it can reach points in non-convex regions of a front, as argued in Learning the Pareto Front with Hypernetworks and A Tale of Two Problems. We deliberately use expected Chebyshev utility rather than hypervolume as the training signal. Hypervolume improvement gets exponentially more expensive as objectives are added, and the expected-utility form lets a stochastic gradient use one sampled $\lambda$ per step. Hypervolume is kept as the evaluation metric.

Research Program

The method reuses the leader–follower machinery of Stackelberg PPO and adds one change. The follower is a single policy that takes $\lambda$ as an input, in the spirit of Pareto hypernetworks, which map a preference to network weights in one model rather than training one model per preference. For each leader update we sample a small batch of preferences. We roll out the conditioned follower on the current hardware and compute the Stackelberg implicit gradient of each sampled Chebyshev utility with respect to $\theta$. As in the original paper, a Fisher-information approximation stands in for the follower Hessian. The update averages these gradients. The PPO-style clipping on hardware changes, which Stackelberg PPO introduced to stop the controller collapsing after a structural edit, is kept unchanged.

For discrete structural edits we keep the original log-derivative estimator. For continuous parameters in SoftZoo we replace it with pathwise gradients from the differentiable simulator.

The analysis has three parts. The first is the flexibility gap. We compute an upper-bound front by running scalar co-design separately at each of a grid of preferences, so every preference gets its own hardware, and take the union. We then report what fraction of that union's hypervolume a single design recovers under each method. The second is a preference-shift test. Hardware is designed under one preference distribution and scored under another, which tells us whether the benefit is robustness or just averaging. The third is a controlled study on a small smooth problem, with one or two continuous hardware parameters and a linear-quadratic follower. There the inner maximization has a closed form, so we can check that the estimated leader gradient matches the exact one before trusting it on the robotics tasks.

The relation to the closest multi-objective work is specific. Parametric Pareto Set Learning for Expensive Multi-Objective Optimization supplies a related tool: a hypernetwork mapping external parameters and preferences to Pareto-optimal decisions, trained with Gaussian-process surrogates. But its external parameters are given contexts to be generalized over, not decisions to be optimized, and it has no controller adapting in closed loop. We optimize over the context, and the thing we optimize is the value of the whole conditioned front. Multi-Objective Bilevel Learning places several objectives at the upper level of a bilevel problem. A Tale of Two Problems flattens bilevel problems into single-level ones using the lower level's stationarity condition. Both constrain our design choices. The first requires a strongly convex lower level, which an RL controller does not have. The second handles only equality constraints and replaces the follower with its first-order condition. That is exactly the move the Stackelberg paper warns against when hardware edits temporarily break the controller. We therefore keep an explicit adapting follower rather than flattening.

Baselines

The naive reference point is sequential design. We take the hardware produced by scalar co-design at a single nominal preference (equal weights). We freeze it, then train the preference-conditioned controller on it for the same number of controller steps our method uses, and report the hypervolume of the resulting front. Beating it means a higher single-design hypervolume with a paired difference across at least five seeds that excludes zero, and no more than a small loss at the nominal preference itself.

The strongest existing co-design method is Stackelberg PPO (Efficient Morphology-Control Co-Design via Stackelberg Proximal Policy Optimization), run with its released settings on the scalarized reward at each preference. Its reported number is a 20.66% average reward gain over BodyGen on the MuJoCo co-design suite. For SoftZoo the reference is the 0.332 co-design score of SoftZoo. Neither paper reports any multi-objective or hypervolume result: we found no reported result on Lacuna for hypervolume on either benchmark, so the multi-objective numbers will be our own.

For fairness, every method gets the same total environment steps, counted across all its runs and summed over preferences for per-preference methods. Every method also gets the same hyperparameter-search budget (an equal number of tuning trials per method) and the same seeds. Every final design is scored by the same procedure, a freshly trained conditioned controller with a fixed step budget. That way no method profits from a better evaluation-time controller.

Two ablations isolate the key ingredient. The first keeps our pipeline but trains the leader on a single fixed $\lambda$, which checks that gains come from optimizing over preferences and not from the conditioned follower alone. The second replaces Chebyshev with a weighted sum, which checks whether reaching non-convex parts of the front matters on these tasks.

Why not just…?

Why not just run scalar co-design at several weights and pick the best design afterwards? This "generate and screen" approach is the most serious threat. It is simple, reuses existing code, and could find a flexible design by luck. We include it with an equal total step budget. For each candidate design from a grid of preferences, we train a conditioned controller on it and keep the design with the best front. If it matches our method, the honest conclusion is that flexibility can be screened for rather than optimized. We would report that, and the flexibility-gap analysis would still stand as a contribution. We expect it to fall short when the most flexible design is a compromise that no single scalar weighting ever produces. The preference-shift test and the flexibility gap will show whether such compromises exist.

Why not just take a weighted sum and use a single-objective co-designer? A weighted sum cannot reach points in non-convex regions of a front. A Tale of Two Problems reports a hypervolume of $2.47 \times 10^{-2}$ for its Chebyshev-based method against $1.66 \times 10^{-5}$ for MoCo on HelpSteer, though that is language-model alignment, not co-design. Whether front concavity matters for speed–energy trade-offs in locomotion is an open empirical question, and the weighted-sum ablation answers it directly. If the fronts turn out to be convex, the simpler scalarization is fine and we will say so.

Why not just use multi-objective Bayesian optimization over the hardware, with an inner controller trained per query? For a handful of continuous parameters this is reasonable, and PPSL-MOBO shows that surrogate-based loops can be sample-efficient. It reports 200 evaluations in total versus 1,000 per configuration for baselines. But each query here costs a full controller training run. Stackelberg PPO's edits are discrete, and GP surrogates scale cubically with data, which is part of why GP-based HEBO took over 21,000 seconds on a 50-dimensional problem in Un-evaluated Solutions May Be Valuable in Expensive Optimization. We run a GP loop that optimizes the same expected Chebyshev utility on SoftZoo's continuous parameters, at equal step budget. This settles whether the gradient-based leader is needed or whether the objective alone is what matters.

Why not just solve an epsilon-constrained problem for each operating mode with a standard simultaneous solver, as process engineers do? For smooth equation-based models like the RPB column this is the right tool, and we do not claim to beat it there. The file's process models are proprietary gPROMS implementations with pilot-validated error of about 11% on capture rate, and we cannot reproduce them. We therefore confine claims to settings where the follower is a learned policy with no stationarity condition to enforce. The linear-quadratic toy is where the two views meet, since there the epsilon-constraint solution is exact and serves as ground truth.

Decisive Experiments

On the MuJoCo co-design suite used by Stackelberg PPO (Crawler, Cheetah, Swimmer, Walker, and the 3D Pusher and Stepper), we pose two objectives: forward progress and negative total actuator effort, using the standard control cost. On SoftZoo, in the ground, ice and ocean biomes with the caterpillar and fish templates, we pose speed against an actuation-energy measure computed from the simulator's actuation signals. The energy measure is our addition, not a SoftZoo metric.

The primary metric is the hypervolume of the single-design front against a fixed reference point, with the flexibility-gap fraction and utility at the nominal preference reported alongside. The decisive check is a paired comparison across seeds against generate-and-screen and against sequential design. The idea succeeds if our method recovers a clearly larger fraction of the per-preference union front on the tasks where hardware visibly mediates the trade-off. It should do so while staying close to the scalar co-designer at the nominal preference. It fails, and we will report it as failing, if generate-and-screen matches it within seed variance on every task.

A secondary check tests the mechanism. Across tasks, the flexibility gain should correlate with how much the per-preference optimal designs differ from one another. If gains appear where the optimal designs barely differ, something other than hardware-mediated trade-offs is driving them.