Variational Inequality Problems (VIPs) are the mathematical backbone of modern equilibrium-seeking systems, from training Generative Adversarial Networks (GANs) to solving complex multi-agent games. Traditionally, the "gold standard" for proving that an algorithm will actually find a solution is the assumption of Lipschitz continuity—the requirement that the operator’s rate of change is strictly bounded.
However, real-world operators often change abruptly or have unknown growth rates, making the Lipschitz constant impossible to estimate. A growing body of research is shifting the focus toward robust convergence: developing algorithms that reach stable solutions even when the operator is merely continuous, non-smooth, or lacks cocoercivity.
Moving Beyond the Lipschitz Constraint
In classical optimization, we often assume that if you move a small distance in the input space, the output of your function won't jump wildly. This is Lipschitz smoothness. In the context of VIPs, Tatjana Chavdarova and Michael Jordan argue in A Primal-Dual Approach to Solving Variational Inequalities with General Constraints that this assumption is often unrealistic.
The core intuition behind the new wave of research is that we can trade global smoothness for algorithmic control. Instead of requiring the problem to be well-behaved, we make the algorithm more cautious. Kha Pham and Pham Duy Khanh demonstrate that by using a dimishing stepsize sequence (like a p-series), the Gradient Projection Method can be forced to converge even if the operator is "merely continuous" or strongly pseudomonotone. The stepsize shrinks fast enough to cancel out any potential volatility in the operator's gradient.
Last-Iterate vs. Ergodic Convergence
Historically, many VI solvers only guaranteed "ergodic" convergence—meaning the average of all your guesses would eventually reach the solution. In practice, this is frustrating because the very last state of your model might be much worse than the average.
Recent breakthroughs have focused on last-iterate convergence, ensuring the final step is the best one. Eduard Gorbunov and Gauthier Gidel established the first $O(1/K)$ last-iterate rates for the Extragradient method under standard assumptions in Extragradient Method: O(1/K) Last-Iterate Convergence for Monotone Variational Inequalities. Crucially, they found that these methods work even without "cocoercivity," a restrictive property that researchers previously thought was mandatory for last-iterate stability.
Solving Hierarchical and Constrained Systems
The frontier of this research involves "nested" problems, where you must solve one VI subject to the constraints of another. Meggie Marschner shows that combining Tikhonov regularization with inertial proximal schemes allows for strong convergence in these hierarchical settings without needing cocoercivity.
Similarly, when constraints are defined by fixed-point sets, Shigeru Iemoto and Hideaki Iiduka have expanded the class of usable operators from "firmly nonexpansive" to "strongly nonexpansive." This allows solvers to handle more complex mathematical constraints while maintaining weak convergence for monotone, hemicontinuous operators.
Summary of the Shift
The transition in the field can be summarized as follows:
- From Lipschitz to Boundedness: We no longer need the operator to be smooth everywhere, just bounded within the area we are searching.
- From Cocoercivity to Monotonicity: We are moving away from restrictive stability requirements toward broader classes of monotone and pseudomonotone operators.
- From Averages to Last-Iterates: Theoretical guarantees are finally catching up to practical needs, ensuring the final output of an algorithm is the actual equilibrium.