Suppose you have a fixed table of past designs, such as molecules, network architectures, or truss geometries, each scored on several goals that pull against each other, and no budget to test anything new. Surrogate models trained to predict each score, then optimized, tend to walk into regions where the surrogate is wrong and confidently optimistic. For the single-objective version of this problem, a strong fix is already known: train the surrogate only to put designs in the right order, and multiply the training signal by building many ranked lists out of the same data. This proposal extends that fix to conflicting objectives. We train one network that takes a design and a trade-off preference as input. For any preference, it orders designs the way that preference would order them. Its training lists are drawn combinatorially over both designs and preferences.
The obvious extension would rank designs by Pareto dominance, the relation "better or equal on every goal, strictly better on one." That extension starves as the number of goals grows. If two designs have independent, continuously distributed scores, the chance that one dominates the other is $2 \cdot 2^{-M}$ for $M$ objectives: 1/2 for two objectives and 1/32 for six. Most pairs are incomparable, so they give a dominance learner no signal at all. Conditioning on a preference turns every incomparable pair into a comparable one under some trade-off. We make the ranking target depend only on each objective's order in the data, which keeps it invariant to any monotone rescaling of the scores, the property that makes ranking surrogates robust in the first place.
Core Claim
In offline multi-objective optimization, a single preference-conditioned listwise ranker over rank-normalized objectives recovers a better Pareto front, measured by hypervolume, than three alternatives: independent regression surrogates, dominance-only ranking surrogates, and independent per-objective ranking surrogates. The advantage should grow with the number of objectives and under monotone warping of the objectives.
The target is built as follows. For each objective $i$, replace the raw score $y_i(x)$ with its empirical quantile $u_i(x) \in [0,1]$ within the offline dataset. For a preference vector $w$ on the positive unit sphere, order designs by the hypervolume scalarization of Qiuyi Zhang, which is a non-linear weighting that, unlike a weighted sum, reaches non-convex parts of the front. Because only the order matters, the exponent in Zhang's form can be dropped:
$$ t_w(x) = \min_{i} \frac{u_i(x)}{w_i}. $$
The surrogate $g_\theta(x, w)$, a network with parameters $\theta$, is trained with ListNet. For each training step we sample a random preference $w$ and a list of $m$ designs $L$. We then minimize the cross-entropy between the softmax of $t_w$ over $L$ and the softmax of $g_\theta(\cdot, w)$ over $L$:
$$ \mathcal{L}(\theta) = \mathbb{E}_{w, L}\Big[-\sum_{x \in L} \operatorname{softmax}_L(t_w)(x) \, \log \operatorname{softmax}_L(g_\theta(\cdot, w))(x)\Big]. $$
The number of distinct training lists scales with both the number of design subsets and the continuum of preferences. That is the combinatorial expansion. At search time, we pick a spread of preferences $w$, run gradient ascent or evolutionary search on $g_\theta(\cdot, w)$ from elite seeds, and pool the results into a front.
Where This Sits in Prior Work
The method comes mostly from Offline Model-Based Optimization by Learning to Rank (RaM). RaM samples ranked lists from a single-objective offline dataset, trains with ListNet or RankCosine, and z-score normalizes outputs for gradient ascent. It reports that surrogate MSE out of distribution has near-zero or negative correlation with final design quality. We reuse its list sampling and losses. Our changes are the preference conditioning and the quantile-based multi-objective target, which RaM never considers.
Ranking over Regression for Bayesian Optimization and Molecule Selection motivates the discrete-space experiments. It shows that pairwise ranking losses rescue graph neural networks on rugged molecular landscapes and that Kendall tau predicts optimization success better than regression $R^2$. Its authors name multi-objective extension as future work.
Two papers already rank directly on Pareto dominance. Dominance-Based Pareto-Surrogate for Multi-Objective Optimization (RASM) trains one Ranking SVM on dominance pairs and reports roughly 1.5x speedups at two objectives and over 2x at ten. It also converges prematurely on ZDT2 and IHR2. Pareto-Wise Ranking Classifier for Multiobjective Evolutionary Neural Architecture Search (CENAS) classifies architectures as dominated or not against clustered reference points, and its authors suggest listwise models as a next step. Both work online inside an evolutionary loop, and both learn only from comparable pairs. We work offline and extract signal from incomparable pairs through the preference input.
Preference-Based Multi-Objective Optimization with Gaussian Process also uses pairwise preferences over trade-offs. It gets them from a live human through dueling bandits, whereas we synthesize preferences from data.
Offline Multi-Objective Optimization supplies both the testbed and the problem. Every surrogate it tests is a regression model. Its best-ranked approach is "Multiple Models + IOM," and it reports that most methods fail to beat the training set on multi-objective combinatorial tasks.
One piece of prior work cuts against us. Design-Bench found that plain CMA-ES and whitened gradient ascent beat elaborate generative methods on 4 of 8 tasks, so extra machinery has to earn its keep. We keep the model to one network and include those plain optimizers as baselines.
Research Program
We will build the preference-conditioned ranker as an MLP, or as a graph or sequence encoder on discrete tasks, with $w$ concatenated to the input. It will plug into the Off-MOO-Bench search loop. Continuous tasks use gradient ascent. Discrete and permutation tasks use NSGA-II-style evolutionary search that queries $g_\theta$.
Three analyses will explain any result rather than just score it. First, we measure the fraction of comparable pairs in each benchmark's data as $M$ varies. This tests whether dominance-only learners are actually signal-starved there. Second, on held-out elite designs, we compute out-of-distribution Kendall tau between $g_\theta(\cdot, w)$ and the true $t_w$. This checks whether ranking fidelity under a preference predicts hypervolume, the multi-objective analogue of the tau-versus-$R^2$ finding of Tom et al. Third, we run a warping stress test that applies strongly non-linear monotone transforms, such as exponentials and heavy-tailed noise, to individual objectives in the offline data. Quantile normalization should leave our method untouched but hurt regression surrogates.
Baselines
The naive reference point is the offline data itself. We report the hypervolume of the training set's own non-dominated front, since Off-MOO-Bench reports that many methods fail to exceed it on combinatorial tasks. We also run a second naive reference: whitened multiple-MSE surrogates with gradient ascent or NSGA-II, the kind of simple baseline Design-Bench found hard to beat.
The strongest existing method is Multiple Models + IOM from Offline Multi-Objective Optimization, which is the best average rank on Off-MOO-Bench. The file gives only that qualitative ranking; no reported per-task hypervolume result was found on Lacuna for Multiple Models + IOM. We will therefore rerun it from the released code on identical splits. Beating it means a better average rank across the suite, plus statistically significant hypervolume wins on a majority of tasks over at least 5 seeds. At minimum it means parity on continuous tasks and a clear win on the combinatorial ones.
The competitor most likely to match us is per-objective RaM: $M$ separate ListNet rankers, one per objective, with their z-scored outputs fed to NSGA-II, which needs only orders, and to weighted gradient ascent. We will also run a modernized RASM, meaning the same network trained only on dominance pairs with a margin loss. For RaM itself, Tan et al. report average ranks of 2.2 (ListNet) and 2.7 (RankCosine) against 5.9 for the best prior method on Design-Bench. We will first reproduce its single-objective behavior as a sanity check.
Every surrogate gets the same architecture, the same 20-trial hyperparameter budget tuned on a held-out validation split, and the same number of search queries. Every surrogate is trained on the same number of ranked lists, so the per-objective baseline sees $M$ times fewer lists per model rather than more total compute.
The ablations isolate the key ingredient:
- Raw z-scored objectives in place of quantiles, which tests invariance.
- A weighted sum in place of the hypervolume scalarization, which tests coverage of non-convex fronts.
- A fixed small grid of preferences in place of freshly sampled ones, which tests whether the combinatorial expansion over preferences matters.
- Pairwise margin loss in place of listwise.
Why not just…?
Why not just train one RaM ranker per objective? This is the real threat, and we treat it as the main comparison. Pareto dominance depends only on per-objective orders, so perfect per-objective rankers determine dominance perfectly. Two things argue against it. First, each ranker's output scale is arbitrary, so combining them for a specific trade-off, as gradient search must, corresponds to no well-defined preference. Second, the search can exploit whichever single ranker extrapolates most optimistically, which Off-MOO-Bench identifies as the source of front collapse. If per-objective RaM with NSGA-II matches us in hypervolume, the result shows that preference conditioning is unnecessary, and we will report it as such. The out-of-distribution Kendall tau under $t_w$ tells us which failure explains the outcome.
Why not just use regression surrogates per objective, the benchmark's best? RaM's finding that out-of-distribution MSE barely correlates with design quality, and the activity-cliff failures reported by Tom et al., suggest regression will be exploited. The warping stress test turns this into a direct measurement rather than an assumption. If regression wins on smooth synthetic tasks, we expect that and will say so. Our claim is about rugged, warped, and discrete tasks.
Why not just rank by Pareto dominance, as RASM and CENAS do? The comparable-pair count falls as $2 \cdot 2^{-M}$ under independence, and RASM's premature convergence suggests that learning only from dominance pairs is brittle. The modernized-RASM baseline and the comparable-pair analysis across $M$ test this. Our prediction is that the gap is small at $M = 2$ and widens by $M \geq 5$.
Why not just scalarize with random weights and fit a regression model? A weighted sum misses non-convex fronts, as Cheng et al. (Pareto Fronts and Incomparable Goals) argue. Regression on scalarized values also inherits sensitivity to scale and noise that comparison-based models avoid, per Comparison-based surrogate models. Our weighted-sum ablation and a regression-on-$t_w$ variant separate the effect of the ranking loss from the effect of the scalarization.
Proposed Experiments
The main study runs on Off-MOO-Bench from Offline Multi-Objective Optimization: the synthetic DTLZ/ZDT families, multi-objective neural architecture search, multi-objective reinforcement learning, the multi-objective TSP tasks, molecule and protein design, and the truss and pressure-vessel engineering problems. We report hypervolume after a fixed budget of final proposals, average rank across tasks, and Kendall tau under held-out preferences. To push the number of objectives higher, we add multi-objective scenarios from YAHPO Gym, whose surrogate benchmark reports rank fidelity to real training of Spearman 0.90 to 0.99, and we subsample its offline data.
Four checks decide the question:
- Beating per-objective RaM in hypervolume at $M \geq 3$, with the margin growing in $M$.
- Exceeding the training-set front on the combinatorial tasks, where Off-MOO-Bench reports that regression-based methods mostly fail.
- Hypervolume unchanged under monotone warping while the regression baselines degrade.
- The fixed-preference-grid ablation losing to random preference sampling, which would show that the combinatorial expansion over preferences, and not the ranking loss alone, is doing the work.
If the first check fails while the others hold, the right conclusion is that per-objective ranking suffices for offline multi-objective optimization. That would still be a new and useful result against the benchmark's regression-only baselines.
References
- Optimal Scalarizations for Sublinear Hypervolume Regret
- Offline Model-Based Optimization by Learning to Rank
- Ranking over regression for Bayesian optimization and molecule selection
- Dominance-Based Pareto-Surrogate for Multi-Objective Optimization
- Pareto-Wise Ranking Classifier for Multiobjective Evolutionary Neural Architecture Search
- Preference-Based Multi-Objective Optimization with Gaussian Process
- Offline Multi-Objective Optimization
- Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization
- Preference-Based Policy Iteration: Leveraging Preference Learning for Reinforcement Learning
- Black-box optimization benchmarking of IPOP-saACM-ES on the BBOB-2012 noisy testbed
- YAHPO Gym - An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimization