Abstract
In order to build intelligent robots to accomplish soccer game tasks, this paper introduces evolutionary computing in agent architecture for perception, planning, and action. An architecture based on Particle Swarm Optimization (PSO) is proposed, consisting of four levels: atomic action, combo action, behavior, and policy. Through offline training, agents develop perception rules and parameters to optimize information gathering regarding position and orientation. PSO is then used to build a set of combo actions described by atomic actions and parameters. Finally, during gameplay, PSO searches for the optimal combination of tasks, behaviors, and combo actions to achieve game objectives. Simulation experiments on the RoboCup2D platform demonstrate that this PSO-based approach is a robust and flexible control method, capable of rapid learning and planning without classical techniques.
Introduction: The Challenge of Robotic Soccer
In the dynamic environment of RoboCup2D, autonomous agents must navigate a complex landscape of split-second decision-making. They must balance high-level strategic goals, such as scoring a goal, with immediate physical constraints like ball velocity and player stamina. Traditional robotic architectures typically fall into two categories: hierarchical (top-down planning) or reactive (bottom-up response).
However, these traditional setups suffer from three major flaws identified by the researchers:
- Knowledge Acquisition Difficulty: Manually coding the preconditions for every possible action in a fluid game is nearly impossible.
- Operational Rigidity: Agents often struggle to adapt to dynamic changes; a shift in top-level policy may not be reflected quickly enough in lower-level actions.
- Layer Isolation: Most systems fail to account for the interplay between layers. A high-level passing strategy is ineffective if the low-level kicking mechanism isn't tuned to the current ball state.
Proposed Architecture
To address these issues, the researchers propose a unified architecture grounded in Nils Nilsson’s "Triple-Tower" model, which separates agent functions into Perception, Model, and Action towers.
Building upon this, the authors introduce a 4-level hierarchy: Atomic Action (basic motor commands), Combo Action (skill sequences), Behavior, and Policy (overall game strategy). The core of this system is Particle Swarm Optimization (PSO), used in two distinct phases:
Offline Learning
During the offline phase, PSO is employed to train the agent's perception and "Combo Actions." For instance, the agent optimizes a "Dribble_ball" combo by testing various sequences of atomic actions—such as dash, turn, and kick—against sparring agents in controlled 2vs2 scenarios. This allows the agent to format robust perception rules and optimize parameters for position and orientation.
Online Strategy and Evolutionary Planning
During a live match, the agent treats decision-making as a multivariable optimization problem. Instead of following a rigid script, the agent uses PSO to evaluate "particles" that represent a unified combination of task, behavior, and action.
By treating the hierarchy of tasks as a particle in a swarm, the agent can rapidly find "good enough" solutions in a complex search space. This ensures that high-level strategy is always grounded in the current capabilities of the low-level motor skills.
Experimental Setup and Datasets
The study utilizes the RoboCup2D simulation platform, a standard AI benchmark where two teams of 11 autonomous agents compete.
- Training Environment: Initial training and sparring were conducted in 2vs2 scenarios (one player and one goalie per side).
- Opponents: To ensure a high standard of learning, the agents were tested against well-known, high-performing teams such as WrightEagle, Gliders, and Oxsy.
Key Results and Insights
The simulation results demonstrated that PSO-based agents are both robust and flexible.
Rapid Learning and Robustness: Agents learned effective behaviors in simulated environments without the need for extensive manual rule-tuning. They displayed complex planning behaviors, such as positioning for a pass, which emerged naturally from the optimization process rather than being explicitly programmed.
Smooth Task Transitions: Quantitative analysis showed that the swarm often "prepares" multiple tasks simultaneously. This allows the agent to switch smoothly between tasks—for example, transitioning from Dribble_pass to Location_arrive—as the game state evolves.
Decomposition and Unified Particles: The researchers found that searching for a global solution in real-time was computationally expensive. By decomposing goals into "Combo Action Frames" (e.g., Hard_kick or Block_robot), they provided a structured framework for PSO to optimize efficiently.
Limitations and Future Outlook
While successful, the approach has limitations. PSO, though faster than many evolutionary methods, still faces hurdles in real-time global optimization, necessitating the use of pre-defined task frames. Furthermore, these "Combo Action Frames" currently require manual specification of parameters. Future work aims to enable agents to autonomously "discover" these frames and to scale the system to full 11vs11 coordination, potentially involving multi-agent PSO.
Summary updated