In the quest for quantum advantage on Noisy Intermediate-Scale Quantum (NISQ) hardware, Variational Quantum Algorithms (VQAs) have become the primary workhorse. These hybrid algorithms use a classical optimizer to tune the parameters of a quantum circuit, much like training a neural network. However, the "landscape" of quantum optimization is notoriously hostile. Traditional gradient-based methods—the bedrock of classical deep learning—often fail here due to two fundamental obstacles: barren plateaus and hardware noise.
Recent research has converged on a robust alternative: replacing gradients with derivative-free optimization strategies, such as evolutionary algorithms, reinforcement learning, and stochastic search.
The Problem: Barren Plateaus and Noisy Slopes
To understand why gradients fail, imagine trying to find the lowest point in a valley while blindfolded, relying only on the slope beneath your feet. In a classical neural network, the slope (gradient) usually points you toward a solution. But in quantum circuits, as the number of qubits increases, the optimization landscape often becomes an exponentially flat desert known as a barren plateau.
As noted by Min-Hsiu Hsieh and Yidong Liao in Quantum optimization for training quantum neural networks, these plateaus render classical gradient-based optimizers stuck in a featureless expanse where the gradient effectively vanishes. Furthermore, physical quantum hardware is "noisy." Every measurement is probabilistic (shot noise), and environmental interference adds jitters to the data. Riccardo Manenti and colleagues at Rigetti Computing found that these inconsistencies make calculating reliable derivatives nearly impossible, necessitating strategies that can navigate noisy objective functions without relying on local slopes Unsupervised Machine Learning on a Hybrid Quantum Computer.
Evolutionary Strategies: Survival of the Fittest Circuits
Evolutionary algorithms bypass the need for gradients by treating the optimization as a "black box." Instead of following a single path down a slope, they maintain a population of candidate solutions and "evolve" them over time.
Lukas Franken and his team demonstrated this with QNEAT, an algorithm that uses evolutionary selection to jointly optimize both the architecture and the parameters of a circuit Gradient-free quantum optimization on NISQ devices. By using discrete mutations—like inserting or deleting a quantum gate—QNEAT can "jump" across flat regions of the landscape that would trap a gradient-descent algorithm.
Similarly, Arthur G. Rattew and Marco Pistoia introduced the Evolutionary Variational Quantum Eigensolver (EVQE). They found that asexual mutation strategies are particularly effective for quantum circuits because the high degree of entanglement makes standard "crossover" (combining parts of two different circuits) mathematically destructive A Domain-agnostic, Noise-resistant Evolutionary Variational Quantum Eigensolver for Hardware-efficient Optimization in the Hilbert Space.
Filtering Noise with Stochastic and Bayesian Models
When gradients are unavailable, how do we choose the next step? Researchers have turned to sophisticated statistical models to "guess" the landscape.
- CMA-ES and SPSA: The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is a popular choice for high-dimensional problems. Hao Wang and Thomas Bäck showed that a well-tuned CMA-ES can outperform standard quantum optimizers like SPSA by using its internal statistical model to "filter" out measurement noise Performance comparison of optimization methods on variational quantum algorithms.
- Bayesian Optimization: This approach builds a surrogate model (often a Gaussian Process) of the quantum cost function. As demonstrated by Riccardo Manenti, this allows the optimizer to suggest new parameters that balance exploring unknown regions of the landscape with exploiting known low-energy states, all without ever calculating a derivative.
Reinforcement Learning and Architecture Search
Beyond just tuning angles of rotation gates, some researchers are using gradient-free methods to design the circuits themselves. This is known as Quantum Architecture Search.
Manwen Liao and Yuxiang Yang developed QCrep, a reinforcement learning framework where a Transformer-based agent learns a policy for placing gates Reinforced Learning Explicit Circuit Representations for Quantum State Characterization from Local Measurements. Because the agent learns a strategy rather than optimizing continuous parameters, it remains trainable even in the complex landscapes where traditional variational methods fail.
This shift toward "learning to optimize" is also seen in multi-agent systems. Michael Kölle showed that evolutionary optimization can train multi-agent quantum reinforcement learning policies that use significantly fewer parameters than classical networks while remaining robust to the barren plateau problem Multi-Agent Quantum Reinforcement Learning using Evolutionary Optimization.
The Future: Fully Quantum Optimization
While most current methods are hybrid (quantum circuit, classical optimizer), a final frontier involves moving the optimization itself onto the quantum processor. Min-Hsiu Hsieh suggests that by using quantum search algorithms and phase oracles, we can identify global optima through quantum interference Quantum optimization for training quantum neural networks. This "maximally quantum" approach would bypass the hybrid communication bottleneck and the barren plateau problem entirely, using quantum mechanics to solve the very problems quantum mechanics created.