Two Fielded Teams and Two Experts: A RoboCup Challenge Response from the Trenches

Milind Tambe, Gal A. Kaminka, Stacy Marsella, Ion Muslea, Taylor Raines

Abstract

The RoboCup (robot world-cup soccer) effort, initiated to stimulate research in multi-agents and robotics, has blossomed into a significant effort of international proportions. RoboCup is simultaneously a fundamental research effort and a set of competitions for testing research ideas. At IJCAI'97, a broad research challenge was issued for the RoboCup synthetic agents, covering areas of multi-agent learning, teamwork and agent modeling. This paper outlines our attack on the entire breadth of the RoboCup research challenge, on all of its categories, in the form of two fielded, contrasting RoboCup teams, and two off-line soccer analysis agents. We compare the teams and the agents to generalize the lessons learned in learning, teamwork and agent modeling.

Introduction

Building autonomous agents capable of real-time cooperation in adversarial environments remains a core challenge in AI. The RoboCup Simulation League provides a "perfect storm" of difficulties: agents must navigate sensor noise, limited stamina, and restricted fields of vision while competing against dynamic opponents. This paper details the development of the ISIS97 and ISIS98 teams, which addressed the IJCAI’97 RoboCup Challenge through three primary pillars: multi-agent learning, robust teamwork models, and post-hoc agent modeling.

Methods and Architecture

The ISIS teams employ a two-tier architecture. A lower-level C layer manages physics and motor control, while the higher-level decision-making is governed by the Soar integrated cognitive architecture. Within Soar, behavior is organized through a hierarchy of "operators," ranging from high-level strategic goals to specific tactical actions.

Hierarchical operator structure in the Soar architecture

To facilitate coordination, the authors implemented STEAM (Shell for TEAMwork), a general-purpose teamwork model based on the theory of joint commitments. STEAM allows agents to autonomously reason about communication needs and reorganization. For skill acquisition, the teams utilized C4.5 Decision Trees for offline shooting analysis and Reinforcement Learning (RL) for online adaptation of ball-interception paths.

Coordination and Analysis

A key innovation in this work is the use of "socially-attentive" monitoring. The TEAMORE agent measures teamwork quality by tracking the synchronization time of sub-teams. Additionally, the ISAAC (ISI Soccer Automated Assistant Coach) tool uses decision trees to identify patterns in match logs, helping researchers find "bugs" in agent behavior, such as poor aim or positioning.

Interestingly, the researchers found that "social learning"—sharing experiences across different roles—could be detrimental. Interception skills, for instance, are highly specialized to a player's role and the specific opponents they face. Furthermore, the older ISIS97 team demonstrated that overlapping player regions could provide "dynamic load balancing," where teammates naturally compete to take over tasks from exhausted or out-of-position peers.

Key Results

The ISIS teams were highly successful, securing 3rd place at RoboCup97 and 4th place at RoboCup98. Quantitative evaluations demonstrated that the STEAM model's communication capabilities significantly improved performance. As shown in the table below, reducing communication costs (allowing for more frequent updates) led to better mean goal differences against top-tier opponents like Andhill97 and CMUnited97.

Impact of communication cost on team performance

Furthermore, the TEAMORE agent confirmed that communication reduced the average time for a team to switch tactics from over 50 time-ticks to under 7, highlighting the efficiency gains of structured coordination.

Conclusion

The success of the ISIS teams underscores the value of domain-independent teamwork models like STEAM, which allowed for the reuse of nearly 45% of coordination rules across different scenarios. While tools like ISAAC and TEAMORE currently serve as "collaborative assistants" requiring human interpretation, they represent a significant step toward fully automated agent modeling and performance analysis in complex multi-agent systems.

Summary updated

Abstract

The RoboCup (robot world-cup soccer) effort, initiated to stimulate research in multi-agents and robotics, has blossomed into a significant effort of international proportions. RoboCup is simultaneously a fundamental research effort and a set of competitions for testing research ideas. At IJCAI'97, a broad research challenge was issued for the RoboCup synthetic agents, covering areas of multi-agent learning, teamwork and agent modeling. This paper outlines our attack on the entire breadth of the RoboCup research challenge, on all of its categories, in the form of two fielded, contrasting RoboCup teams, and two off-line soccer analysis agents. We compare the teams and the agents to generalize the lessons learned in learning, teamwork and agent modeling.
BibTeX
@inproceedings{tambe1999two,
  author = {Milind Tambe and Gal A. Kaminka and Stacy Marsella and Ion Muslea and Taylor Raines},
  title = {{Two Fielded Teams and Two Experts: A RoboCup Challenge Response from the Trenches}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1999},
}
Figure