Complex Momentum for Optimization in Games

Jonathan P. Lorraine, David Acuna, Paul Vicol, David Duvenaud

Abstract

We generalize gradient descent with momentum for optimization in differentiable games to have complex-valued momentum. We give theoretical motivation for our method by proving convergence on bilinear zero-sum games for simultaneous and alternating updates. Our method gives real-valued parameter updates, making it a drop-in replacement for standard optimizers. We empirically demonstrate that complex-valued momentum can improve convergence in realistic adversarial games–like generative adversarial networks– by showing we can find better solutions with an almost identical computational cost. We also show a practical complex-valued Adam variant, which we use to train BigGAN to improve inception scores on CIFAR-10.

Introduction

In the landscape of deep learning, optimization is traditionally viewed as a single-player journey toward a global minimum. However, many modern architectures, such as Generative Adversarial Networks (GANs), frame optimization as a competitive game between multiple players (e.g., a generator and a discriminator). Standard momentum-based optimizers often struggle in these settings because they are designed for static, real-valued loss landscapes. In differentiable games, the Jacobian of the gradient vector field frequently exhibits imaginary eigenvalues, leading to "rotational" dynamics where players orbit each other rather than converging to a stable Nash equilibrium.

Methodology: Simultaneous Complex Momentum (SimCM)

To counteract these instabilities, we propose Simultaneous Complex Momentum (SimCM). By generalizing the momentum coefficient $\beta$ to be complex-valued ($\beta \in \mathbb{C}$), we allow the momentum buffer to oscillate. This oscillation acts as a "counter-rotation" that cancels out the natural spiraling behavior of adversarial competition.

Figure 4: Trajectory of the Generator and Discriminator in a simple game (top) and the corresponding distance to the optimum over iterations (bottom). Complex momentum (orange) successfully converges while classical momentum (blue) fails.

While the internal calculations utilize complex arithmetic, the parameter updates remain real-valued by taking the real part of the complex buffer. This makes SimCM a drop-in replacement for standard optimizers in frameworks like PyTorch or JAX. We further extend this logic to create Complex Adam, where the $\beta_1$ parameter is generalized to the complex plane.

Experimental Evaluation

The effectiveness of complex momentum was tested across a variety of scenarios:

  • Bilinear Zero-Sum Games: Providing a foundation for theoretical convergence proofs.
  • Dirac-GAN: A simplified model used to visualize and analyze rotational dynamics.
  • 2D Mixture of Gaussians: A synthetic task for evaluating generative performance.
  • CIFAR-10: Training a full-scale BigGAN to demonstrate practical scalability and performance gains.

Key Results and Insights

Our experiments revealed that a "sweet spot" exists for the complex phase of $\beta$, which we term almost-positive momentum (typically $\arg(\beta) \approx \pi/8$). This setting provides the acceleration benefits of standard momentum in cooperative regions while maintaining the stability required for adversarial regions.

Figure 6: Comparison of gradient evaluations required for convergence across different levels of game "adversarialness" ($\gamma_{max}$). Complex momentum (orange) shows superior robustness compared to classical momentum and competitive performance with Extragradient (EG).

Key findings include:

  • Convergence Stability: On the Dirac-GAN, complex momentum converges smoothly to the optimum, whereas standard momentum and SGD often diverge.
  • Improved GAN Performance: Using Complex Adam to train BigGAN on CIFAR-10 resulted in an Inception Score (IS) of 9.25, outperforming the baseline of 9.10.
  • Robustness: Complex momentum is less sensitive to the degree of "adversarialness" in a game compared to classical momentum and performs competitively with more complex first-order methods like Extragradient (EG).
  • Efficiency: The computational overhead is minimal. While memory usage for the buffer doubles, the number of expensive gradient evaluations remains unchanged.

Discussion and Future Work

Complex momentum introduces a new hyperparameter: the phase angle of $\beta$. While $\pi/8$ is a strong default, further research could explore automated tuning or player-specific phases to account for the differing spectral properties of generators and discriminators. Additionally, fully integrating complex logic into Adam's second-moment estimation remains an open question for future optimization research.

Summary updated

Abstract

We generalize gradient descent with momentum for optimization in differentiable games to have complex-valued momentum. We give theoretical motivation for our method by proving convergence on bilinear zero-sum games for simultaneous and alternating updates. Our method gives real-valued parameter updates, making it a drop-in replacement for standard optimizers. We empirically demonstrate that complex-valued momentum can improve convergence in realistic adversarial games–like generative adversarial networks– by showing we can find better solutions with an almost identical computational cost. We also show a practical complex-valued Adam variant, which we use to train BigGAN to improve inception scores on CIFAR-10.
BibTeX
@inproceedings{lorraine2022complex,
  author = {Jonathan P. Lorraine and David Acuna and Paul Vicol and David Duvenaud},
  title = {{Complex Momentum for Optimization in Games}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2022},
  url = {https://openreview.net/forum?id=_Zalktn44Z},
}
Figure