Understanding Dynamics of Adam in Zero-Sum Games: An ODE Approach

Yi Feng, Weiming Ou, Xiao WangICML 2026 · OpenReview

Abstract

The remarkable success of the Adam in training neural networks has naturally led to the widespread use of its descent-ascent counterpart, Adam-DA, for solving zero-sum games. Despite its popularity in practice, a rigorous theoretical understanding of Adam-DA still lags behind. In this paper, we derive ordinary differential equations (ODEs) that serve as continuous-time limits of the Adam-DA. These ODEs closely approximate the discrete-time dynamics of Adam-DA, providing a tractable analytical framework for understanding its behavior in zero-sum games. Using this ODE approach, we investigate two fundamental aspects of Adam-DA: local convergence and implicit gradient regularization. Our analysis reveals that the roles of the first- and second-order momentum parameters in zero-sum games are exactly the opposite of their well-documented effects in minimization problems. We validate these predictions through GAN experiments across multiple architectures and datasets, demonstrating the practical implications of this reversed momentum effect.
BibTeX
@inproceedings{feng2026understanding,
  author = {Yi Feng and Weiming Ou and Xiao Wang},
  title = {{Understanding Dynamics of Adam in Zero-Sum Games: An ODE Approach}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=4MVVscCjYu},
}

Loading related content...

Figure