Understanding Dynamics of Adam in Zero-Sum Games: An ODE Approach

Yi Feng, Weiming Ou, Xiao Wang

Introduction

While the Adam optimizer has become the undisputed workhorse for training deep neural networks in standard minimization tasks (like image classification or regression), its behavior in zero-sum games remains notoriously difficult to tame. In min-max scenarios—such as Generative Adversarial Networks (GANs) or adversarial training—the optimization landscape is not a stable bowl but a shifting, competitive battlefield. Practitioners frequently observe that the standard "best practices" for Adam's momentum parameters fail in these environments, sometimes even finding that negative momentum yields better results.

Despite these empirical observations, a rigorous theoretical explanation for Adam's divergent behavior in games has been missing. This paper bridges that gap by deriving a high-fidelity continuous-time model (an Ordinary Differential Equation, or ODE) for the Adam Descent-Ascent (Adam-DA) algorithm. Through this framework, the authors uncover a fundamental "momentum reversal": to achieve stability and find desirable "flat" solutions in zero-sum games, practitioners must use the exact opposite parameter settings compared to standard minimization.

The Continuous-Time ODE Framework

To decode the mechanics of Adam-DA, the researchers treat the algorithm's discrete steps as a continuous flow, employing an ODE approach to model its trajectory.

  • High-Resolution Modeling: Moving beyond previous rough approximations, the authors derive a "Continuous Adam-DA" model with a highly accurate local error of $O(h^3)$. This precision captures subtle optimization dynamics that simpler models overlook.
  • Jacobian Analysis for Local Convergence: By analyzing the eigenvalues of the system's Jacobian matrix at equilibrium, the researchers can mathematically pinpoint which combinations of step sizes and momentum values lead to stable convergence versus chaotic divergence.
  • Implicit Gradient Regularization (IGR): The ODE framework allows the authors to examine the algorithm's "hidden" biases. They demonstrate how Adam-DA implicitly minimizes or maximizes the gradient norm, effectively steering the training trajectory toward specific types of equilibria.

Key Insights: The Momentum Reversal

The theoretical analysis yields several surprising discoveries that upend conventional wisdom regarding Adam's hyperparameters:

  • The Great Momentum Flip: In standard minimization, a larger first-order momentum ($\beta$) generally accelerates and stabilizes training. In zero-sum games, the mathematics dictate the opposite: a smaller $\beta$ (and even negative $\beta$) significantly improves local convergence and broadens the range of stable step sizes.
  • Flipped Flatness: Optimizers that find "flat" regions of the loss landscape are highly desirable because they tend to generalize better. While minimization requires large $\beta$ and small second-order momentum ($\rho$) to find these flat minima, zero-sum games require small $\beta$ and large $\rho$ to guide the trajectory toward flatter equilibria.
  • The Bilinear Dead-End: The paper establishes a sobering corollary: on simple bilinear games (the fundamental "Hello World" of min-max problems), Adam-DA always diverges, regardless of hyperparameter tuning. This exposes a structural limitation of the algorithm in purely competitive, unregularized environments.
  • The Role of $\epsilon$: Typically, the $\epsilon$ parameter in Adam is treated merely as a tiny constant to prevent division by zero. The ODE analysis reveals that in games, a larger $\epsilon$ actively acts as a stabilizer, enhancing local convergence.

Empirical Validation

To ensure these theoretical findings hold up in practice, the authors conducted extensive GAN training experiments across multiple datasets (CIFAR-10, STL-10, and CelebA) and architectures (standard CNNs and ResNets).

  • Theoretical Match: The derived Continuous Adam-DA ODE almost perfectly tracks the actual discrete algorithm's path in experiments, far outperforming previous continuous models like SignGDA-flow.
  • Stability Bounds: The authors provide a concrete formula for the maximum stable step size $h$. They empirically confirm that as $\beta$ increases, the "stability window" for $h$ shrinks, explaining why GANs are so notoriously brittle with high momentum.
  • Performance Gains: When using the parameter settings predicted to find "flatter" regions (small/negative $\beta$, large $\rho$), the GANs consistently achieved higher Inception Scores (IS) and lower FID scores. For instance, on a ResNet trained on CIFAR-10, the Inception Score jumped from ~4.16 (with $\beta=0.5$) to ~7.08 (with $\beta=-0.3$), indicating significantly higher quality and diversity in generated images.

Limitations and Future Directions

While this paper provides a robust mathematical foundation for Adam-DA, the authors note a few limitations. The core analysis is deterministic; although the GAN experiments utilize stochastic mini-batches, the theoretical proofs do not yet fully account for stochastic noise. Additionally, the findings are most applicable to "interaction-dominated" games (like GANs) where competition drives the dynamics, and may not apply as neatly to games that resemble two independent minimization problems.

Looking forward, the authors suggest extending this high-resolution ODE framework to multi-player general-sum games, paving the way for better optimization strategies in complex multi-agent reinforcement learning environments.

Summary updated

Abstract

The remarkable success of the Adam in training neural networks has naturally led to the widespread use of its descent-ascent counterpart, Adam-DA, for solving zero-sum games. Despite its popularity in practice, a rigorous theoretical understanding of Adam-DA still lags behind. In this paper, we derive ordinary differential equations (ODEs) that serve as continuous-time limits of the Adam-DA. These ODEs closely approximate the discrete-time dynamics of Adam-DA, providing a tractable analytical framework for understanding its behavior in zero-sum games. Using this ODE approach, we investigate two fundamental aspects of Adam-DA: local convergence and implicit gradient regularization. Our analysis reveals that the roles of the first- and second-order momentum parameters in zero-sum games are exactly the opposite of their well-documented effects in minimization problems. We validate these predictions through GAN experiments across multiple architectures and datasets, demonstrating the practical implications of this reversed momentum effect.
BibTeX
@inproceedings{feng2026understanding,
  author = {Yi Feng and Weiming Ou and Xiao Wang},
  title = {{Understanding Dynamics of Adam in Zero-Sum Games: An ODE Approach}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=4MVVscCjYu},
}
Figure