Introduction
While the Adam optimizer has become the undisputed workhorse for training deep neural networks in standard minimization tasks (like image classification or regression), its behavior in zero-sum games remains notoriously difficult to tame. In min-max scenarios—such as Generative Adversarial Networks (GANs) or adversarial training—the optimization landscape is not a stable bowl but a shifting, competitive battlefield. Practitioners frequently observe that the standard "best practices" for Adam's momentum parameters fail in these environments, sometimes even finding that negative momentum yields better results.
Despite these empirical observations, a rigorous theoretical explanation for Adam's divergent behavior in games has been missing. This paper bridges that gap by deriving a high-fidelity continuous-time model (an Ordinary Differential Equation, or ODE) for the Adam Descent-Ascent (Adam-DA) algorithm. Through this framework, the authors uncover a fundamental "momentum reversal": to achieve stability and find desirable "flat" solutions in zero-sum games, practitioners must use the exact opposite parameter settings compared to standard minimization.
The Continuous-Time ODE Framework
To decode the mechanics of Adam-DA, the researchers treat the algorithm's discrete steps as a continuous flow, employing an ODE approach to model its trajectory.
- High-Resolution Modeling: Moving beyond previous rough approximations, the authors derive a "Continuous Adam-DA" model with a highly accurate local error of $O(h^3)$. This precision captures subtle optimization dynamics that simpler models overlook.
- Jacobian Analysis for Local Convergence: By analyzing the eigenvalues of the system's Jacobian matrix at equilibrium, the researchers can mathematically pinpoint which combinations of step sizes and momentum values lead to stable convergence versus chaotic divergence.
- Implicit Gradient Regularization (IGR): The ODE framework allows the authors to examine the algorithm's "hidden" biases. They demonstrate how Adam-DA implicitly minimizes or maximizes the gradient norm, effectively steering the training trajectory toward specific types of equilibria.
Key Insights: The Momentum Reversal
The theoretical analysis yields several surprising discoveries that upend conventional wisdom regarding Adam's hyperparameters:
- The Great Momentum Flip: In standard minimization, a larger first-order momentum ($\beta$) generally accelerates and stabilizes training. In zero-sum games, the mathematics dictate the opposite: a smaller $\beta$ (and even negative $\beta$) significantly improves local convergence and broadens the range of stable step sizes.
- Flipped Flatness: Optimizers that find "flat" regions of the loss landscape are highly desirable because they tend to generalize better. While minimization requires large $\beta$ and small second-order momentum ($\rho$) to find these flat minima, zero-sum games require small $\beta$ and large $\rho$ to guide the trajectory toward flatter equilibria.
- The Bilinear Dead-End: The paper establishes a sobering corollary: on simple bilinear games (the fundamental "Hello World" of min-max problems), Adam-DA always diverges, regardless of hyperparameter tuning. This exposes a structural limitation of the algorithm in purely competitive, unregularized environments.
- The Role of $\epsilon$: Typically, the $\epsilon$ parameter in Adam is treated merely as a tiny constant to prevent division by zero. The ODE analysis reveals that in games, a larger $\epsilon$ actively acts as a stabilizer, enhancing local convergence.
Empirical Validation
To ensure these theoretical findings hold up in practice, the authors conducted extensive GAN training experiments across multiple datasets (CIFAR-10, STL-10, and CelebA) and architectures (standard CNNs and ResNets).
- Theoretical Match: The derived Continuous Adam-DA ODE almost perfectly tracks the actual discrete algorithm's path in experiments, far outperforming previous continuous models like SignGDA-flow.
- Stability Bounds: The authors provide a concrete formula for the maximum stable step size $h$. They empirically confirm that as $\beta$ increases, the "stability window" for $h$ shrinks, explaining why GANs are so notoriously brittle with high momentum.
- Performance Gains: When using the parameter settings predicted to find "flatter" regions (small/negative $\beta$, large $\rho$), the GANs consistently achieved higher Inception Scores (IS) and lower FID scores. For instance, on a ResNet trained on CIFAR-10, the Inception Score jumped from ~4.16 (with $\beta=0.5$) to ~7.08 (with $\beta=-0.3$), indicating significantly higher quality and diversity in generated images.
Limitations and Future Directions
While this paper provides a robust mathematical foundation for Adam-DA, the authors note a few limitations. The core analysis is deterministic; although the GAN experiments utilize stochastic mini-batches, the theoretical proofs do not yet fully account for stochastic noise. Additionally, the findings are most applicable to "interaction-dominated" games (like GANs) where competition drives the dynamics, and may not apply as neatly to games that resemble two independent minimization problems.
Looking forward, the authors suggest extending this high-resolution ODE framework to multi-player general-sum games, paving the way for better optimization strategies in complex multi-agent reinforcement learning environments.
Summary updated