In standard machine learning, optimization is a solitary journey. A single model adjusts its parameters to minimize a loss function, sliding smoothly down a gradient landscape toward a local minimum. But in modern architectures like Generative Adversarial Networks (GANs), adversarial training, and multi-agent reinforcement learning, optimization becomes a competitive game. Two or more players—such as a generator and a discriminator—constantly adjust their strategies to outmaneuver one another.
This competitive dynamic introduces a fundamental mathematical headache: rotational fields. Instead of settling into a neat minimum, simultaneous gradient updates frequently cause players to chase each other in endless, expanding circles.
While optimization theorists traditionally analyzed simultaneous updates because the math was cleaner, practitioners noticed a curious pattern: forcing players to take turns—using alternating updates—consistently stabilized training. A deep body of research now explains exactly why alternating updates fundamentally alter the geometry of competitive optimization, transforming divergent spirals into stable paths to equilibrium.
The Core Failure of Simultaneous Updates
To understand why alternating updates work, we must first look at why simultaneous updates fail. Consider a simple zero-sum bilinear game: $\min_x \max_y x^T A y$. The ideal solution is a saddle point at the origin.
If both players compute their gradients simultaneously based on the current state $(x_t, y_t)$ and update their parameters at the exact same moment, they act on outdated information. By the time player $x$ moves to where player $y$ was, player $y$ has already moved somewhere else. As Gidel et al. demonstrate in Negative Momentum for Improved Game Dynamics, simultaneous gradient updates are inherently divergent in these bilinear settings; they introduce an artificial outward push that forces the system to spiral away from the equilibrium.
This lack of coordination fails to account for the sequential structure inherent to minimax games, as noted by Kim et al. in Double-Step Alternating Extragradient with Increasing Timescale Separation for Finding Local Minimax Points: Provable Improvements. When players update simultaneously, the system behaves like a naive reactive loop, leading to severe instability.
Simultaneous Updates:
Player 1 (uses state t) ----> New State 1
Player 2 (uses state t) ----> New State 2
Result: Overshooting and outward spiraling (Divergence)
Alternating Updates:
Player 1 (uses state t) ----> New State 1
|
v
Player 2 (uses New State 1) --> New State 2
Result: Error cancellation and stabilization (Convergence)
Mechanism 1: The Informational Advantage and Error Cancellation
Alternating updates break this destructive symmetry by introducing a sequential dependency. Player 1 updates their parameters first, and Player 2 immediately uses this fresh parameter state to calculate their own gradient.
This simple shift provides a powerful informational advantage. As explored by Cho et al. in SGDA with shuffling: faster convergence for nonconvex-PŁ minimax optimization and Lee et al. in Fundamental Benefit of Alternating Updates in Minimax Optimization, sequential updates drastically reduce the computational penalty imposed by the cross-interactions between competing variables.
From a mathematical perspective, this turn-taking enables a higher-order cancellation of error terms. Majka et al. prove in Mirror Descent-Ascent for mean-field min-max problems that because the second player adjusts within the very same iteration, the resulting error terms scale far more favorably with the step size than they do in simultaneous schemes. This error cancellation acts as a stabilizing anchor, allowing the optimization trajectory to stay on course.
Mechanism 2: Symplectic Discretization and Energy Preservation
In continuous time, competitive game dynamics can be modeled as smooth differential equations (skew-gradient flows). These continuous paths often form stable, closed orbits around an equilibrium—much like a frictionless pendulum swinging back and forth, conserving its total energy.
When we run an algorithm on a computer, we must discretize this continuous flow into distinct steps.
- Simultaneous updates act like a standard Euler discretization. In physics simulations, using Euler's method on a pendulum adds artificial energy to the system at every step, causing the pendulum to swing wider and wider until it flies apart.
- Alternating updates act as a symplectic integrator. Symplectic integrators are specialized numerical methods designed to preserve the underlying geometry and volume of a state space.
In Alternating Mirror Descent for Constrained Min-Max Games, Wibisono et al. show that treating discrete updates as a symplectic discretization ensures the system preserves a modified energy function. Instead of flying outward, the strategies stay tightly bound to the ideal continuous trajectory.
Feng formalizes this in Prediction Accuracy of Learning in Games : Follow-the-Regularized-Leader meets Heisenberg by establishing a Heisenberg-type uncertainty principle for game dynamics. While simultaneous updates trigger an exponential explosion of prediction error, alternating updates preserve the volume of the state space, maintaining long-term stability and predictability.
Mechanism 3: Implicit Damping and Suppressing Rotations
The rotational components of a gradient field are the primary drivers of oscillation in minimax problems. If you visualize the gradient vectors as a whirlpool, simultaneous updates blindly follow the swirling current outward.
Alternating updates introduce a hidden mathematical gift: an implicit correction term. In Continuous-Time Analysis of Heavy Ball Momentum in Min-Max Games, Feng et al. demonstrate that taking turns injects a friction-like damping force directly into the optimization path. This implicit damping actively saps energy from the oscillatory components of the system.
Remarkably, this structural stabilization renders explicit, memory-intensive optimistic gradient methods redundant. The authors of Near-optimal Local Convergence of Alternating Gradient Descent-Ascent for Minimax Optimization prove that alternating updates naturally suppress the rotational components of the gradient field. By neutralizing these rotations, alternating updates allow the system to safely use larger step sizes, unlocking a quadratic speedup—achieving a near-optimal $O(\kappa)$ local convergence rate compared to the sluggish $O(\kappa^2)$ rate of simultaneous updates.
Quantifiable Speedups Across Domains
The practical benefits of alternating updates are backed by rigorous convergence guarantees across a wide variety of optimization landscapes:
Bilinear and Zero-Sum Games
In constrained bilinear environments, Nan et al. prove in On the $O(1/T)$ Convergence of Alternating Gradient Descent–Ascent in Bilinear Games that alternating updates achieve a fast $O(1/T)$ convergence rate. This is a massive theoretical leap over the standard $O(1/\sqrt{T})$ rate observed in simultaneous methods, confirming that sequential updates provide a fundamental speedup. Similarly, Zhang et al. show in Convergence of Gradient Methods on Bilinear Zero-Sum Games that alternating updates provide significantly larger stability regions than joint optimization.
Nonconvex Optimization and the PL Condition
When moving beyond convex settings, alternating methods remain highly robust. Under the two-sided Polyak-Łojasiewicz (PL) condition, Yang et al. demonstrate in Global Convergence and Variance Reduction for a Class of Nonconvex-Nonconcave Minimax Problems and Faster Single-loop Algorithms for Minimax Optimization without Strong Concavity that alternating GDA prevents cyclical divergence and converges efficiently to global optima without requiring strict concavity assumptions.
Multi-Agent Learning and Voting Systems
The power of sequential updates extends well beyond gradient descent. In discrete multi-agent environments, Cai et al. show in Last-Iterate Convergence Properties of Regret-Matching Algorithms in Games that alternating strategy updates drastically reduce the oscillatory behavior of algorithms like Regret Matching+, explaining why it powers superhuman poker AIs. Even in social choice theory, Meir et al. prove in Convergence to Equilibria in Plurality Voting that while simultaneous voting changes lead to endless loops, sequential best-response dynamics are guaranteed to converge to a stable Nash equilibrium.
Distributed and Federated Settings
In decentralized networks, adversarial training is plagued by client drift and communication delays. Tarzanagh et al. establish in FedNest: Federated Bilevel, Minimax, and Compositional Optimization that alternating between inner and outer variables provides crucial stability in federated minimax problems, successfully suppressing the oscillations typical of decentralized adversarial training.
Conclusion
The empirical preference for alternating updates is not a fluke or a mere engineering hack. By shifting from simultaneous to sequential updates, optimization algorithms fundamentally change how information flows through a competitive system.
By granting the second player an immediate look-ahead advantage, alternating updates unlock higher-order error cancellation, act as energy-preserving symplectic integrators, and introduce implicit damping forces that suppress destabilizing rotations. Whether training a GAN, optimizing a federated model, or solving a complex zero-sum game, letting players take turns remains one of our most mathematically elegant defenses against divergence.
Key Foundational Work
Convergence to Equilibria in Plurality Voting (2010) — contributed 1 concept elements to this direction.
Symmetric Strategy Improvement (2015) — contributed 1 concept elements to this direction.