Abstract
Traditional optimization methods rely on explicit mathematical constraints, which are often unavailable in complex real-world applications like drug discovery or robotics. In these scenarios, constraints are typically represented by a dataset of feasible examples. We introduce DiffOPT, a framework that treats optimization as a constrained sampling problem. By utilizing diffusion models to learn the feasible manifold from data, DiffOPT reformulates the task as sampling from a "Product of Experts" distribution. Our results across diverse benchmarks, including DesignBench and molecular design, demonstrate that DiffOPT consistently finds high-quality solutions that remain within the implicit boundaries of the data distribution.
Introduction: The Challenge of Unknown Constraints
In standard optimization, we minimize an objective function $h(x)$ subject to known constraints (e.g., $x \in C$). However, many practical problems involve "black-box" or implicit constraints. For instance, in molecular design, a molecule must not only be potent but also synthesizable. Since there is no simple formula for synthesizability, we must rely on databases of known molecules to define the feasible region.
Ignoring these implicit constraints often leads to "spurious solutions"—designs that optimize the objective but are physically or biologically impossible. DiffOPT addresses this by asking: Can we use existing data to learn the boundaries of feasibility and stay within them during optimization?
Methodology: DiffOPT as a Product of Experts
DiffOPT reformulates optimization as a sampling problem. Instead of a point estimate, it seeks to sample from a target density $\pi_\beta(x)$ that balances the objective and the constraints: $$\pi_\beta(x) \propto p(x) \exp[-\beta h(x)]$$ Here, $p(x)$ is the data distribution (the "feasible manifold") learned by a diffusion model, and $\exp[-\beta h(x)]$ is the Boltzmann distribution representing the objective function.
Sampling Strategies
Depending on the differentiability of the objective $h(x)$, DiffOPT employs two distinct strategies:
- Differentiable Objectives (Two-Stage Sampling):
- Stage I (Warm-up): Uses "guided diffusion" to move particles toward regions with high data density and favorable objective scores.
- Stage II (Correction): Employs Langevin Dynamics (a type of MCMC) to refine the samples. This stage is crucial for correcting the inherent biases of guided diffusion, ensuring convergence to the true product distribution.
- Non-differentiable Objectives (Iterative Importance Sampling): When gradients are unavailable, DiffOPT uses an iterative process. It proposes particles via the diffusion model, weights them by the objective, and "diversifies" the top candidates by adding and subsequently removing noise through the diffusion process.
Experimental Setup
The framework was evaluated on several challenging tasks:
- Synthetic Branin Task: A 2D visualization task with a custom oval-shaped feasible region.
- DesignBench: A suite of six offline black-box optimization tasks, including material design (Superconductor), DNA sequence affinity (TFBind), and robot morphology (Ant/D’Kitty).
- Multi-objective Molecule Optimization: Optimizing drug-likeness (QED), synthesizability (SA), and enzyme activity (GSK3B) in a latent space.
Results
DiffOPT demonstrates superior performance across all benchmarks. In the DesignBench suite, it achieved an average rank of 2.0, outperforming state-of-the-art baselines like DDOM and COMs. Notably, in the Superconductor task, DiffOPT improved performance by 9.6% over the next best competitor.
In molecular optimization, DiffOPT not only achieved the highest objective scores but also maintained the highest percentage of valid molecules. This highlights the model's ability to respect the learned manifold even when pushed toward extreme objective values. Furthermore, the framework proved highly sample-efficient, maintaining top performance even with limited inference budgets.
Discussion and Limitations
While DiffOPT is a powerful tool for constrained optimization, it faces some challenges. Guided diffusion can occasionally drift from the trained manifold, potentially leading to error accumulation. Future work may involve incorporating explicit manifold constraints to mitigate this. Additionally, like other model-based approaches, DiffOPT's performance is tied to the accuracy of the surrogate model used for the objective; for example, lower performance on the "Ant" dataset was attributed to surrogate training difficulties.
Conclusion
DiffOPT provides a robust solution for optimization problems where constraints are defined by data. By leveraging the manifold-learning capabilities of diffusion models and a principled two-stage sampling approach, it bridges the gap between high-performance optimization and realistic, feasible design.
Summary updated