Situated Vision in a Dynamic World: Chasing Objects

Ian Horswill, Rodney A. BrooksAAAI 1988

Abstract

This paper presents a reactive vision system for a mobile robot designed to track and follow moving objects in real-time. Moving away from traditional 3D reconstruction paradigms, the authors utilize a "situated vision" approach where low-resolution visual modules are linked directly to motor controls. By prioritizing local heterogeneity over global homogeneity and employing a subsumption-style architecture, the system achieves robust "approach-and-follow" behavior in cluttered, dynamic environments without the need for complex internal world models.

Introduction

In the late 1980s, computer vision was largely synonymous with "reconstruction"—the slow, computationally expensive process of building a complete 3D geometric model of a scene before deciding how to move. This approach struggled in dynamic, "messy" environments where lighting changes and objects move unpredictably.

Horswill and Brooks challenge this paradigm by asking if a robot can interact with a changing world using "situated vision." Instead of building a map, they propose a reactive system where vision serves as a direct pipeline to action. Their specific goal was to implement an approach-and-follow behavior: identifying an "interesting" moving object and maintaining a set distance from it, effectively letting the object "drag" the robot through the environment.

System Architecture

The system is built using a "subsumption-style" architecture—a network of small, asynchronous processes. It was implemented on the MIT AI Lab’s "Allen" robot, powered by a Lisp machine.

Block diagram of the robot's control architecture, showing the flow from visual input to motor commands.

Motor commands are derived directly from image coordinates. A "centroid" box finds the object's center; the "rotate" box turns the robot to center the object, and the "advance" box drives the motors to maintain a specific vertical position in the visual field, which correlates to distance on a flat floor. A "boredom" mechanism resets the system if the tracker loses the object for too long, allowing the robot to seek new stimuli.

The Vision Pipeline

To achieve real-time performance, the system subsamples video to a resolution of only 32 x 28 pixels. The vision pipeline relies on several key technical innovations:

  • Local Heterogeneity Segmentation: Instead of looking for regions of uniform color (homogeneity), which often "leak" into the background due to noise, the system looks for "heterogeneity" (areas of change or texture). It then uses a "blob coloring" algorithm to group these into visual objects.
  • The Matcher & Latch: To track an object across frames, a "matcher" compares current blobs to a "latch" (the silhouette of the object from the previous frame). This assumes the object will not move more than its own width between frames.
The vision processing stages: raw low-resolution input (left), heterogeneity segmentation (center), and the tracked object silhouette (right).

Key Insights and Results

The most profound insight of this work is the synergy of unreliable parts. While the motion detector and segmenter are individually "poor," their errors do not overlap in ways that break the system when mediated by a situated control network. By running at 5 frames per second, the robot treats the world as its own model, reacting to where an object is right now rather than remembering where it was in 3D space.

Key Results include: * Real-Time Performance: The loop rate was fast enough to track a one-foot object moving at up to five feet per second. * Robustness: The robot successfully tracked objects against complex backgrounds, including shiny linoleum floors with strong reflections. * Emergent Behavior: Because the robot follows at a "respectful distance" and backs away if an object gets too close, human observers often described the robot as "playing," despite the code having no formal concept of play.

Limitations and Future Work

The system's primary limitation is its lack of general collision avoidance; the robot only "sees" the object it is chasing and may crash into walls or furniture if led there. Additionally, the simple segmentation can fail if the tracked object moves too close to a background element, causing them to merge into a single blob. The authors suggest that future iterations could include "whiskers" for physical sensing or dedicated modules for corridor following to improve navigation in complex spaces.

Summary updated

BibTeX
@inproceedings{horswill1988situated,
  author = {Ian Horswill and Rodney A. Brooks},
  title = {{Situated Vision in a Dynamic World: Chasing Objects}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1988},
}
Figure