Optimization in the real world—whether designing a stable walking gait for a robot, a new life-saving molecule, or an efficient aircraft wing—usually happens in high-dimensional spaces. Traditional methods like Bayesian Optimization (BO) are the gold standard for these "black-box" problems because they are sample-efficient, but they suffer from a fatal flaw: the curse of dimensionality. As the number of variables grows, the volume of the search space increases exponentially, making it impossible for a surrogate model to provide meaningful guidance.
A growing body of research suggests a powerful workaround: instead of optimizing in the raw, high-dimensional input space, we should optimize within a learned, low-dimensional latent manifold.
The Geometry of Feasibility
In many engineering and scientific problems, the "true" dimensionality of the solution is much lower than the representation suggests. For example, an image of a molecule might have thousands of pixels, but the underlying chemical properties are governed by a few dozen structural parameters.
By using generative models like Variational Autoencoders (VAEs) or Generative Adversarial Networks (GANs), researchers can compress these complex spaces. Rika Antonova and colleagues demonstrated in Bayesian Optimization in Variational Latent Spaces with Dynamic Compression that by "compressing" undesirable regions of a latent space, an optimizer can treat diverse failure modes as a single area. This allows robots to learn complex tasks like manipulation in as few as 10 to 20 trials, rather than the thousands typically required.
This approach effectively "unfolds" the search space. In Black-Box Optimization with Implicit Constraints for Public Policy, Wenqian Xing and Chong Liu show that latent manifolds can capture "implicit constraints"—rules that are hard to write as equations but easy for experts to recognize. By mapping only the feasible solutions into a latent space, the optimizer never wastes time evaluating designs that are physically or socially impossible.
Smoothing the Landscape
Optimization is easiest when the "landscape" of the objective function is smooth. Raw design spaces are often jagged, discrete, or riddled with "adversarial artifacts"—points that look good to a model but are actually failures.
Latent spaces act as a natural regularizer. Cashen Diniz and Mark Fuge introduced Least Volume Analysis, a method that forces autoencoders to find the intrinsic topological dimension of a dataset. This reduction smoothens the performance landscape, leading to more stable convergence for downstream optimization.
Similarly, in the realm of symbolic regression, Qian Li and colleagues developed GenSR, which maps the chaotic, discrete world of mathematical equations into a continuous latent space. This allows an evolution strategy to "glide" through the space of possible formulas to find the simplest and most accurate one, rather than jumping blindly between discrete expressions.
Jointly Learning the Map and the Goal
The most recent shift in this field is moving away from "pre-trained" latent spaces toward jointly learned representations. If the latent space is learned independently of the optimization goal, it might capture the wrong features—focusing on how a design looks rather than how it performs.
Natalie Maus and Maximilian Balandat addressed this with Joint Composite Latent Space Bayesian Optimization (JoCo). JoCo jointly trains neural network encoders and Gaussian Process surrogates, ensuring the latent space is specifically compressed to highlight task-relevant features. This allows the model to handle "grey-box" problems where high-dimensional intermediate data (like an image of a fluid simulation) is available before the final performance score is known.
To ensure these maps remain reliable, Junyu Xuan and Yingzhen Li proposed HiBBO, which enforces geometric consistency. By aligning the latent representation with the actual landscape of the objective function, they ensure that points that are close in the latent space actually perform similarly in the real world.
From Engineering to AI Steering
The implications of latent space optimization extend far beyond traditional engineering. Cyril Picard and Faez Ahmed describe this as Generative Optimization (GenOpt)—a new paradigm that hybridizes the creative speed of Generative AI with the rigor of optimization.
We see this in Personalized Image Generation via Human-in-the-loop Bayesian Optimization, where Rajalaxmi Rajagopalan uses BO to navigate the latent space of a diffusion model. Instead of a user struggling to write the perfect prompt, the system iteratively optimizes the latent features based on user feedback, "steering" the generative model toward a specific mental image.
By treating high-dimensional problems as searches along a low-dimensional, learned manifold, researchers are turning "impossible" optimization tasks into manageable ones, bridging the gap between raw data and optimal design.