Tight Regret Upper and Lower Bounds for Optimistic Hedge in Two-Player Zero-Sum Games

Taira Tsuchiya

Abstract

In two-player zero-sum games, the learning dynamic based on optimistic Hedge achieves one of the best-known regret upper bounds among strongly-uncoupled learning dynamics. With an appropriately chosen learning rate, the social and individual regrets can be bounded by $O(\log(mn))$ in terms of the numbers of actions $m$ and $n$ of the two players. This study investigates the optimality of the dependence on $m$ and $n$ in the regret of optimistic Hedge. To this end, we begin by refining existing regret analysis and show that, in the strongly-uncoupled setting where the opponent's number of actions is known, both the social and individual regret bounds can be improved to $O(\sqrt{\log m \log n})$. In this analysis, we express the regret upper bound as an optimization problem with respect to the learning rates and the coefficients of certain negative terms, enabling refined analysis of the leading constants. We then show that the existing social regret bound as well as these new social and individual regret upper bounds cannot be further improved for optimistic Hedge by providing algorithm-dependent individual regret lower bounds. Importantly, these social regret upper and lower bounds match exactly including the constant factor in the leading term. Finally, building on these results, we improve the last-iterate convergence rate and the dynamic regret of a learning dynamic based on optimistic Hedge, and complement these bounds with algorithm-dependent dynamic regret lower bounds that match the improved bounds.

Introduction

In game theory and machine learning, a central goal is to enable agents to learn a Nash equilibrium through "uncoupled" dynamics. In this setting, each player learns only from their own observed rewards without explicit knowledge of their opponent's strategy. Optimistic Hedge has emerged as a state-of-the-art algorithm for this task, renowned for its fast convergence properties.

Despite its success, a significant theoretical gap remained. While it was established that social regret (the sum of players' regrets) could be bounded by $O(\log(mn))$, the optimality of this bound—specifically regarding the dependence on the number of actions ($m$ and $n$) and the leading constant factors—was not fully understood. This research addresses these fundamental limits: Can the performance of Optimistic Hedge be further improved, and what are its absolute performance boundaries?

Methodology

The study employs a rigorous theoretical framework combining refined regret analysis with optimization techniques.

Refined Regret Analysis and Optimization

The authors generalize the standard upper bound for Optimistic Hedge by introducing parameters that capture the trade-off between different negative terms in the regret formula. Rather than selecting learning rates ad hoc, they treat the regret bound as an optimization problem. This approach allows for the identification of mathematically optimal learning rates tailored to specific objectives, such as minimizing social versus individual regret.

Visualization of the trade-off between individual regret components ($f^*, g^*$) and the social regret objective ($J^*$).

Cardinality-Awareness

A key innovation is the introduction of "cardinality-aware" dynamics. In this setting, players are assumed to know the number of actions available to their opponent, even if the opponent's strategy remains unknown. This additional information allows for more precise tuning of learning rates, leading to tighter bounds.

Lower Bound Construction

To establish optimality, the authors construct specific "hard" payoff matrices and analytically derive the minimum regret any Optimistic Hedge agent must incur on such instances. This proves that the derived upper bounds are essentially tight.

Key Results and Insights

The primary contribution of this work is the establishment of tight regret bounds that close the gap between theory and practice for Optimistic Hedge.

Improved Bounds through Cardinality-Awareness

The most striking finding is that cardinality-awareness enables a tighter bound of $O(\sqrt{\log m \log n})$. This represents a major improvement over the standard $O(\log m + \log n)$ bound, particularly in highly asymmetric games where one player has significantly more moves than the other.

Comparison of regret upper and lower bounds for standard and cardinality-aware learning dynamics. Note: $M = \ln m$ and $N = \ln n$.

Exact Constants and Trade-offs

Unlike many theoretical bounds that ignore constant factors, this work matches upper and lower bounds exactly for social regret, including the leading constant factors. Furthermore, the optimization framework reveals an inherent Pareto frontier: it is impossible to simultaneously minimize both players' individual regrets to their absolute minimums without a trade-off.

Dynamic Regret and Convergence

The findings extend to dynamic regret (performance against a changing best strategy) and last-iterate convergence. The authors show that using time-averaged strategies is nearly optimal, improving the dynamic regret bound to $O(\sqrt{\log m \log n} \log T)$, the best known for uncoupled dynamics.

Conclusion and Future Work

This paper provides the first tight regret analysis for Optimistic Hedge in two-player zero-sum games, proving that cardinality-aware learning rates achieve essentially optimal performance.

While social regret bounds are now perfectly tight, a small gap remains for individual regret. Future research could focus on closing this gap and extending these tight bounds to multi-player general-sum games or more complex regret measures like swap regret. Additionally, while these lower bounds are algorithm-dependent for Optimistic Hedge, determining if any uncoupled dynamic can surpass these limits remains an open question.

Summary updated

Abstract

In two-player zero-sum games, the learning dynamic based on optimistic Hedge achieves one of the best-known regret upper bounds among strongly-uncoupled learning dynamics. With an appropriately chosen learning rate, the social and individual regrets can be bounded by $O(\log(mn))$ in terms of the numbers of actions $m$ and $n$ of the two players. This study investigates the optimality of the dependence on $m$ and $n$ in the regret of optimistic Hedge. To this end, we begin by refining existing regret analysis and show that, in the strongly-uncoupled setting where the opponent's number of actions is known, both the social and individual regret bounds can be improved to $O(\sqrt{\log m \log n})$. In this analysis, we express the regret upper bound as an optimization problem with respect to the learning rates and the coefficients of certain negative terms, enabling refined analysis of the leading constants. We then show that the existing social regret bound as well as these new social and individual regret upper bounds cannot be further improved for optimistic Hedge by providing algorithm-dependent individual regret lower bounds. Importantly, these social regret upper and lower bounds match exactly including the constant factor in the leading term. Finally, building on these results, we improve the last-iterate convergence rate and the dynamic regret of a learning dynamic based on optimistic Hedge, and complement these bounds with algorithm-dependent dynamic regret lower bounds that match the improved bounds.
BibTeX
@inproceedings{tsuchiya2026tight,
  author = {Taira Tsuchiya},
  title = {{Tight Regret Upper and Lower Bounds for Optimistic Hedge in Two-Player Zero-Sum Games}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2026},
  url = {https://openreview.net/forum?id=ZQjGrbZt0H},
}
Figure