Many expensive design problems are shared by organizations that will not share their data. An airframe maker and an engine supplier co-design a part. A drug developer and a partner lab trade off potency against a toxicity assay the partner treats as proprietary. Several teams merge one model, and each team scores it on a validation set it cannot release. In each case the goal is a Pareto front: the set of designs where no objective can improve without another getting worse. What makes these cases hard is that each objective can only be measured by the party that owns it, and each measurement costs hours. Standard multi-objective Bayesian optimization assumes one agent sees every objective's history. Federated learning, where clients train a shared model without pooling raw data, has so far either assumed cheap gradients or targeted a single objective.
This note proposes a protocol in which no party ever reveals its measurements. Each party instead publishes a random draw of its own objective model, written in a shared set of random features so that the draws line up across parties. A coordinator combines one draw per party into a joint guess of all the objectives, computes the Pareto set of that guess, and hands it back. Each party then evaluates, on its own schedule, whichever design in that set its own model is least sure about. The claim is that this decoupled, asynchronous Thompson-sampling protocol gets close to the hypervolume of a centralized optimizer that pools all raw data, with tunable differential privacy and communication that does not grow with the number of evaluations.
Core Claim
Existing work that joins expensive multi-objective optimization and federated learning splits the data "horizontally": every client holds evaluations of the same objectives. FDD-MOEA trains a radial-basis-function surrogate on each client and averages them after sorting their centers. Its authors acknowledge that this breaks when clients cover different regions of the design space, because the centers stop meaning the same thing. Federated Thompson Sampling and its private variant DP-FTS-DE solve a related alignment problem by expressing each Gaussian process (a probabilistic regression model with calibrated uncertainty) in random Fourier features drawn from a common seed. Every client's model then lives in the same weight space. Those methods, however, optimize a single objective.
We take the random-feature trick from FTS and use it for a different job: assembling a joint posterior over objectives held by different parties. We then add the ingredient that expense makes necessary, namely decoupled, asynchronous evaluation. Party $m$ keeps a Bayesian linear model over shared features $\phi(x)$ and publishes a draw
$$\tilde f_m(x) = \phi(x)^\top \omega_m, \qquad \omega_m \sim \mathcal N(\mu_m, \Sigma_m) + \text{DP noise},$$
where $x$ is a design, $\mu_m, \Sigma_m$ are the party's private posterior mean and covariance over feature weights, and the noise is calibrated as in DP-FTS-DE. The coordinator runs NSGA-II on the cheap vector $(\tilde f_1, \dots, \tilde f_M)$ to get a sampled Pareto set. This is the posterior-sampling idea behind qPOTS, here fed by parties that never pool data. When party $m$ is free, it picks from the latest sampled Pareto set the design where its own local posterior variance is highest, evaluates it, updates privately, and publishes a fresh draw. No party waits for any other. The coordinator never sees a measured value, only designs and noisy weight draws.
Research Program
The build has three parts. The first is the protocol above, implemented with a common feature seed, per-party Bayesian linear regression, and NSGA-II on the sampled objectives.
The second is a privacy analysis. Repeated Thompson draws average toward the posterior mean, so leakage accumulates over rounds. We will account for it with the user-level differential-privacy composition used in DP-FTS-DE, and we will also measure it empirically. A coordinator-side attacker will try to reconstruct a party's objective from its published draws, and we will report how accurately it succeeds.
The third is a study of when decoupling pays off. Parties often have very different costs. In transformer architecture search, FLOPs are nearly free while error requires training. Letting the cheap party evaluate often should concentrate the expensive party's budget where the front is uncertain. We will vary cost ratios and each party's latency to find where asynchronous decoupling beats lock-step rounds, which the federated black-box optimization survey identifies as the default assumption that fails under churn.
The known limitation is dimensionality. Random features need many basis functions in high-dimensional design spaces. We therefore restrict claims to design spaces of roughly 2 to 30 dimensions and report the degradation as dimension grows, rather than hiding it.
Proposed Experiments
We will use three benchmark families.
The first is synthetic. We take DTLZ1–DTLZ7 with 3 and 5 objectives, assign each objective to a separate party, and report inverted generational distance (IGD) and hypervolume. DTLZ7's disconnected front is where FDD-MOEA struggles, so it is the key stress test. Branin-Currin and the Four Bar Truss serve as small engineering sanity checks.
The second is a real private-scoring task: "federated model merging." We use the 8-task vision merging suite (SUN397, MNIST, Stanford Cars, RESISC45, EuroSAT, SVHN, GTSRB, DTD) on ViT-B/32. The design is the vector of merging coefficients, and each task owner scores merged models on a validation set it keeps private. This is exactly the setting MAP solves centrally with a quadratic surrogate.
The third has heterogeneous costs: the bi-objective error-versus-FLOPs architecture search from FSEO, where the two objectives are owned by two parties with very different evaluation costs.
Across all three, metrics are hypervolume against the number of expensive evaluations per party and against simulated wall-clock time, bytes communicated, the privacy budget $\epsilon$, and the reconstruction attacker's error.
Baselines
Naive reference. Each party uses its budget on quasi-random (Sobol) designs, and the Pareto front is taken from the pooled results after the fact. A second naive reference has each party optimize only its own objective, which recovers only the extreme points of the front. Our method must clearly beat both at equal per-party budgets.
Centralized upper bound. All raw evaluations are pooled at one server running qPOTS-style Pareto Thompson sampling with full Gaussian processes. It uses the same initial designs, per-party budgets, and hyperparameter tuning budget as our method. We found no reported result on Lacuna for qPOTS on DTLZ or merging benchmarks, so we will run it ourselves. Our target for "beating the privacy cost" is reaching at least 90% of its final hypervolume at $\epsilon < 10$, the privacy regime DP-FTS-DE reports as achievable for single-objective tuning.
Strongest domain method on the merging task. Centralized MAP, which fits a quadratic surrogate from 30–60 merged-model evaluations and reports recovering the full 2-task front with "only 60 evaluations, compared to 2,500 for a comparable evolutionary search," with surrogate $R^2 > 0.95$. We will run MAP with its evaluation budget matched to ours. We will also report Pareto Merging's 85.9% average 8-task accuracy as a centralized, gradient-based reference point that we do not expect to exceed. Beating the MAP baseline means matching its hypervolume within the same number of evaluations while never pooling validation scores.
Strongest federated surrogate method. FDD-MOEA, adapted so each party ships its trained radial-basis-function surrogate to the coordinator. On 3-objective DTLZ it reports the best IGD in 16 of 28 instances against its comparison methods. Beating it means lower IGD on DTLZ7 and comparable IGD elsewhere at matched evaluations, while revealing strictly less, which we will check with the same reconstruction attack.
Scalarized baseline. RFMS-style ParEGO. Parties return scalar metrics to the server, which scalarizes them with random weights. This is the simplest existing cross-site protocol.
Ablations. These isolate the key ingredients: coupled evaluation (every party evaluates the same design each round) versus decoupled; synchronous versus asynchronous; random-feature function draws versus draws on a fixed candidate pool; and DP noise swept from zero upward.
Why not just…?
Why not just send the raw measurements to a trusted server? In the target settings, the measurements themselves are the proprietary asset. That is the premise behind privacy-driven federated work such as Rueckert et al. and the restricted curators in RFMS. Where pooling is allowed, it is the right answer. That is why it appears as our upper bound: the experiments measure exactly how much hypervolume privacy costs. If the gap is large at useful $\epsilon$, the protocol is not worth it, and we will say so.
Why not just have the coordinator send a candidate pool and have each party return a posterior draw on it? This is a real threat. It needs no random features and avoids their high-dimensional weakness. The candidate-pool ablation settles it. We expect the function-draw version to win when fronts are thin or disconnected, as on DTLZ7, because a fixed pool misses them. We expect the pool version to match or win in higher dimensions. If the pool version wins everywhere, we will adopt it and keep the decoupled asynchronous scheduling as the contribution.
Why not just have each party ship its trained surrogate, as FDD-MOEA does? A shipped surrogate hands the coordinator the party's full estimate of its objective at every round. It also gives no principled uncertainty: FDD-MOEA substitutes the spread across clients' predictions, which is undefined when only one party owns each objective. The reconstruction attack and the DTLZ7 comparison test both points directly.
Why not just scalarize and let each party report scores, as RFMS does? Scalar reports still reveal every measured value to the server. Random linear weightings also struggle to reach concave parts of the front, a limitation that motivates Pareto hypernetworks. ParEGO is a baseline on every benchmark, so its hypervolume shortfall, or lack of one, will be measured directly.
Decisive Checks
The idea succeeds if three things hold. On the merging suite, with $\epsilon < 10$, decoupled asynchronous sampling reaches at least 90% of centralized qPOTS hypervolume within MAP's evaluation budget. On DTLZ7, it clearly beats adapted FDD-MOEA in IGD. On the architecture-search task, decoupling measurably saves expensive error evaluations relative to the coupled ablation.
The idea fails in a specific way if the candidate-pool variant dominates the function-draw variant. In that case the random-feature contribution is not needed, and the paper narrows to the scheduling result.
References
- A federated data-driven evolutionary algorithm for expensive multi-/many-objective optimization
- Federated Bayesian Optimization via Thompson Sampling
- Differentially Private Federated Bayesian Optimization with Distributed Exploration
- qttPOTS: Efficient Batch Multiobjective Bayesian Optimization via Pareto Optimal Thompson Sampling
- FSEO: Few-Shot Evolutionary Optimization via Meta-Learning for Expensive Multi-Objective Optimization
- Collaborative and Federated Black-box Optimization: A Bayesian Optimization Perspective
- MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation
- Pareto Merging: Multi-Objective Optimization for Preference-Aware Model Merging
- High Dimensional Restrictive Federated Model Selection with Multi-objective Bayesian Optimization over Shifted Distributions
- Model-Based and Data-Driven Strategies in Medical Image Computing
- Learning the Pareto Front with Hypernetworks