Constrained Consensus-Based Optimization and Numerical Heuristics for the Few Particle Regime

Jonas Beddrich, Enis Chenchene, Massimo Fornasier, Hui Huang, Barbara Wohlmuth

Introduction

Consensus-Based Optimization (CBO) is a versatile multi-particle optimization method designed for nonconvex and nonsmooth global optimization in high dimensions. Inspired by collective behavior in nature, CBO employs a swarm of particles that explore the objective landscape, moving toward a "consensus point"—a weighted average biased toward regions of lower energy. While global convergence has been proven for unconstrained problems, many real-world applications in physics and engineering involve complex constraints.

Conceptual map of Consensus-Based Optimization (CBO) and its relationship to other optimization heuristics and the proposed enhancements.

This work addresses two critical gaps: the lack of rigorous convergence theory for constrained domains and the high computational cost associated with the "many-particle" requirement ($N \to \infty$) typically needed for theoretical guarantees. By introducing constrained heuristics (CH) and enhanced strategies (ES), the authors bridge the gap between classical metaheuristics and efficient numerical solvers.

Theoretical Framework for Constrained Domains

The authors extend CBO to handle constrained global optimization on both compact and unbounded domains with boundaries. By leveraging Skorokhod Stochastic Differential Equations (SDEs) with reflecting boundary conditions, they provide the first rigorous proof of global convergence in the many-particle regime.

A key theoretical insight is that the presence of a boundary does not destroy the "exponential-in-time contractivity" of the CBO dynamics. The algorithm remains robust even when the global minimizer resides exactly on the boundary. The paper provides explicit error bounds showing how the error decreases as a function of the time step ($\Delta t$), particle count ($N$), and desired accuracy ($\varepsilon$).

Heuristics for the Few-Particle Regime

To bridge the gap between theory and practical efficiency, the paper introduces numerical heuristics that allow CBO to succeed with significantly fewer particles than previously thought possible.

Enhanced Exploitation and Exploration

  1. Enhanced Exploitation: An adaptive region control mechanism is introduced. By projecting particles onto a shrinking ball centered at the consensus point, the swarm is forced to focus its search as it approaches a potential solution, accelerating convergence.
  2. Enhanced Exploration: In the few-particle regime, the swarm is prone to getting trapped in local minima. The authors propose setting the noise volatility $\sigma$ higher than the theoretical stability limit ($2\lambda$). This "over-noising" prevents premature stagnation and maintains exploration capability despite low particle counts.

Hierarchical Structured Noise

For scientific computing applications, standard Gaussian noise can be inefficient. The authors integrate CBO with a multigrid finite element method, utilizing a hierarchical noise structure that respects the underlying geometry of the problem. This approach is particularly effective for high-dimensional variational problems where the search space is discretized.

Experimental Evaluation and Results

The proposed methods were evaluated against standard benchmarks and complex physical models:

  • High-Dimensional Benchmarks: The enhanced CBO successfully minimized the Rastrigin function in dimensions up to $d=100$. This is a significant improvement over previous CBO implementations, which often struggled beyond $d=20$ due to the curse of dimensionality.
  • Scientific Computing: The algorithm was applied to a constrained $p$-Allen–Cahn problem with obstacles, a challenging phase-transition model from materials science. The hierarchical CBO solved this 127-dimensional problem using only 2,540 particles, demonstrating its viability for complex variational tasks.

Conclusion

This work significantly advances the state of Consensus-Based Optimization by providing both a solid theoretical foundation for constrained problems and practical heuristics for high-dimensional optimization with few particles. While the current theory is limited to convex domains, numerical results suggest broader applicability to non-convex regions. Future work may explore formal proofs for these heuristics and the implementation of adaptive covariance matrices to further refine the exploration process.

Summary updated

Abstract

Consensus-based optimization (CBO) is a versatile multi-particle optimization method for performing nonconvex and nonsmooth global optimizations in high dimensions. Proofs of global convergence in probability have been achieved for a broad class of objective functions in unconstrained optimizations. In this work we adapt the algorithm for solving constrained optimizations on compact and unbounded domains with boundary by leveraging emerging reflective boundary conditions. In particular, we close a relevant gap in the literature by providing a global convergence proof for the many-particle regime comprehensive of convergence rates. On the one hand, for the sake of minimizing running cost, it is desirable to keep the number of particles small. On the other hand, reducing the number of particles implies a diminished capability of exploration of the algorithm. Hence numerical heuristics are needed to ensure convergence of CBO in the few-particle regime. In this work, we also significantly improve the convergence and complexity of CBO by utilizing an adaptive region control mechanism and by choosing geometry-specific random noise. In particular, by combining a hierarchical noise structure with a multigrid finite element method, we are able to compute global minimizers for a constrained $p$-Allen-Cahn problem with obstacles, a very challenging variational problem.
BibTeX
@article{beddrich2026constrained,
  author = {Jonas Beddrich and Enis Chenchene and Massimo Fornasier and Hui Huang and Barbara Wohlmuth},
  title = {{Constrained Consensus-Based Optimization and Numerical Heuristics for the Few Particle Regime}},
  journal = {J. Glob. Optim.},
  year = {2026},
  doi = {10.1007/s10898-026-01612-x},
}
Figure