Reactive Deliberation: An Architecture for Real-Time Intelligent Control in Dynamic Environments

Michael K. SahotaAAAI 1994

Introduction

In the high-stakes, fast-moving world of robot soccer, intelligent control requires a tight, asynchronous loop between "deciding" and "doing." This paper, originally presented at the 1994 AAAI proceedings, introduces Reactive Deliberation, a robot architecture designed to bridge the gap between high-level reasoning and real-time physical reaction. By allowing multiple "behaviours" to compete for control of an "executor" through a distributed bidding system, the system enables robots to navigate unpredictable environments far more effectively than traditional sequential planners.

The Action Selection Problem

In the early 1990s, robotics research was polarized. "Good Old Fashioned AI" (GOFAIR) relied on exhaustive planning that often failed in dynamic settings where the environment changed faster than the planner could compute. Conversely, purely reactive architectures, such as Rodney Brooks’ Subsumption Architecture, offered speed but lacked the representational depth needed for complex, goal-oriented tasks.

The core challenge is the action selection problem: deciding "what to do now" when the world is in constant flux. Author Michael K. Sahota argues that what to do cannot be separated from how to do it; feasibility and utility must be calculated simultaneously and frequently to ensure the robot remains relevant to its environment.

The Reactive Deliberation Architecture

The proposed architecture splits the robot’s control system into two asynchronous components: the Deliberator and the Executor.

The Reactive Deliberation architecture, showing the asynchronous interaction between the Deliberator and the Executor.
  1. The Executor: Acting as the "muscle memory," the Executor consists of action schemas. These are high-speed programs handling specific tasks like "follow path" or "servo to ball." Only one schema runs at a time, maintaining a tight feedback loop with the environment.
  2. The Deliberator: This "decision-maker" comprises multiple concurrent behaviours (e.g., Shoot, Defend Goal, Clear Ball). These behaviours engage in inter-behaviour bidding: each evaluates the current state, plans a potential action, and submits a "bid" representing its expected utility. The behaviour with the highest bid wins, sending its chosen action schema and parameters to the Executor.

Crucially, these layers run asynchronously. While the Executor keeps the robot moving, the Deliberator constantly re-evaluates whether a different goal has become more urgent, allowing for immediate interruptions when opportunities arise.

Experimental Setup: The Dynamite Testbed

The architecture was evaluated using the Dynamite Testbed at the University of British Columbia. This pioneering setup featured:

  • Hardware: 1/24 scale radio-controlled racing cars.
  • Vision: An overhead camera system tracking robots and the ball at 60 Hz.
  • Processing: A "remote brain" approach where off-board computers handled computation and transmitted commands to the cars.
  • Benchmark: The "LCI Cup," a tournament of one-on-one soccer games.

Key Insights and Results

A central finding of the research is the necessity of internal models. Contrary to the "no-representation" movement of the time, Sahota demonstrates that models of robot and ball dynamics are essential for high-speed operation (up to 1 m/s), as they allow the robot to predict future states.

The primary advantage of Reactive Deliberation lies in frequent evaluation. The architecture was compared against two simpler controllers in a round-robin tournament:

  • Half-wit: A sequential controller that only deliberates when an action finishes.
  • No-wit: A basic controller that switches between offense and defense on a fixed timer.
Tournament results comparing Reactive Deliberation against simpler controller architectures.

The results indicate that Reactive Deliberation nearly doubles the scoring output of the sequential "Half-wit" approach. The ability to interrupt ongoing actions the moment a better opportunity arises proved decisive in the dynamic soccer environment.

Limitations and Future Work

While successful, the current implementation has limitations. Bidding criteria are currently hand-tuned, suggesting a need for more formal or learned utility estimation. Additionally, the architecture focuses on one-on-one play and does not yet address multi-robot coordination or complex perceptual processing. Future work aims to incorporate machine learning to allow robots to refine their utility estimates and dynamic models through experience.

Summary updated

BibTeX
@inproceedings{sahota1994reactive,
  author = {Michael K. Sahota},
  title = {{Reactive Deliberation: An Architecture for Real-Time Intelligent Control in Dynamic Environments}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1994},
}
Figure