Improving Dribbling, Passing, and Marking Actions in Soccer Simulation 2D Games Using Machine Learning

Nader Zare, Omid Amini, Aref Sayareh, Mahtab Sarvmaili, Arad Firouzkouhi, Stan Matwin, Amílcar SoaresRobocup 2021 · DOI

TL;DR

This paper details the machine learning strategies behind CYRUS, the RoboCup 2021 2D Soccer Simulation League champions. They introduce novel approaches to dribbling, passing prediction, and multi-agent marking, significantly improving their team's performance and demonstrating the power of advanced AI in complex, dynamic environments.

What Problem Does This Paper Address?

The RoboCup 2D Soccer Simulation League presents a challenging environment for AI agents. It's a stochastic, partially observable world where 24 autonomous agents must coordinate to score goals while defending their own. Key challenges include:

  • Dribbling: Effectively maneuvering the ball while evading opponents and finding optimal passing/shooting opportunities.
  • Passing: Predicting teammate actions to enable seamless collaboration and create scoring chances, especially amidst environmental noise and partial observability.
  • Marking: Developing a robust defensive strategy that synchronizes multiple agents to cover opponents effectively, avoiding issues like multiple agents marking one opponent or leaving others unmarked.

Methods

The CYRUS team implemented several novel machine learning techniques to address these challenges:

  • Multi-Action Dribble (MAD): This strategy uses a Deep Neural Network (DNN) to predict opponent movements, allowing the dribbling agent to find better positions. MAD generates three types of one-step actions: "Two-Step Kick" (to distract opponents), "Move Before First Kick" (to reposition around the ball), and "Turn Before First Kick" (to open up new dribbling angles).
  • Pass Prediction: To improve teammate collaboration, CYRUS developed a "full state action predictor" that forecasts a ball-owner's action even with noisy observations. This involves engineering features related to the ball, own players, and opponents, and training a DNN to predict pass actions.
  • Improved Marking Decision: CYRUS moved beyond simple greedy marking strategies like "Proximity-Based Marking" and "Danger-Based Marking." They developed "Optimized Multi-Agent Matching" (OMAM), which combines ideas from danger-based marking and multi-agent matching. OMAM aims to reduce the number of candidate solutions by considering "K-top tasks" and grouping players (Back, Middle, Forward) and opponents (Attacker, Normal) to create a more synchronized and efficient defensive assignment.

Datasets

The paper mentions data collection for training their DNN models by running games between CYRUS and other teams (e.g., YuShan and Persepolis). Specific details on publicly released datasets are not provided, but they refer to their own "Cyrus 2014 Source" and a "Pyrus - Python 2D Soccer Simulation Base" for future development.

Key Insights and Ideas

  • DNNs for Predictive Dribbling: Using DNNs to predict opponent movements significantly enhances dribbling effectiveness by allowing agents to proactively position themselves.
  • The Importance of "Full State" Prediction: Even in a partially observable environment, training models on "full state" data (simulating noise-free observations) can lead to more robust predictions for actions like passing.
  • Beyond Greedy Marking: Simple greedy approaches to marking are insufficient in dynamic, multi-agent environments. Sophisticated matching algorithms like OMAM are crucial for effective team-level defense.
  • Feature Engineering is Key: The success of their pass prediction model hinges on carefully engineered features that capture crucial game state information.
  • The Trade-off of MAD Alone: Interestingly, using MAD without the DNN predictor actually decreased their winning rate, highlighting the necessity of predictive capabilities for advanced strategies.

Key Results

  • Dribbling: The MAD strategy, especially when combined with the DNN predictor, led to increased winning probabilities against teams like YuShan and Persepolis.
  • Pass Prediction: The proposed features and models achieved pass prediction accuracy rates ranging from 76.23% to 80.51% in pure (noise-free) environments and showed significant improvement over previous methods when noise was present.
  • Marking: OMAM improved CYRUS's defense, reducing the average goals conceded from 0.9 per game in 2019 to 0.33 in 2021.

Limitations and Future Work

  • Limitations: The paper acknowledges that the "full state action predictor" is trained on simulated noise-free observations, which is not the real-world scenario of the RoboCup competitions. The Hungarian Method for marking also suffers from issues like requiring an equal number of agents and tasks and being susceptible to observational noise.
  • Future Work: The authors plan to release the first official version of their Python-based simulation base, PYRUS, which aims to open new avenues for applying machine learning techniques in 2D Soccer Simulation.

Summary updated

BibTeX
@inproceedings{zare2021improving,
  author = {Nader Zare and Omid Amini and Aref Sayareh and Mahtab Sarvmaili and Arad Firouzkouhi and Stan Matwin and Amílcar Soares},
  title = {{Improving Dribbling, Passing, and Marking Actions in Soccer Simulation 2D Games Using Machine Learning}},
  booktitle = {Robocup},
  year = {2021},
  doi = {10.1007/978-3-030-98682-7\_28},
}
Figure