Projective Visualization: Acting from Experience

Marc GoodmanAAAI 1993

Abstract

This paper introduces Projective Visualization (PV), a technique that allows autonomous agents to predict the outcomes of their actions by "simulating" the future using a library of raw, concrete past experiences. By comparing iterative step-by-step simulation against direct long-term prediction, we demonstrate that AI can learn to perform as well as hand-coded reactive systems simply by observing them. Our results highlight a critical trade-off between short-term simulation fidelity and long-term internal consistency.

Introduction

Traditional Case-Based Reasoning (CBR) often relies on abstract representations—semantic features hand-picked by human experts. This creates a "knowledge engineering bottleneck" and risks losing vital information that an expert might deem irrelevant but is actually crucial for success. Projective Visualization (PV) addresses this by building projections using concrete representations (raw sensor data) rather than human-defined abstractions. This approach is presented as a computational form of human mental imagery—using past "visual" experiences to simulate future possibilities.

Methodology

The PV architecture consists of several layers designed to handle raw perceptual data:

  1. Concrete Representation: Instead of abstract concepts, the system stores raw perceptual data (e.g., Cartesian coordinates of joints).
  2. Case Projection: The system builds a library of experiences and induces discrimination trees (using algorithms like ID3) to predict future feature values based on the current state.
  3. Projection Strategies:
  4. Projective Simulation (PS): An iterative approach that predicts the state at $T+1$, then uses that prediction to predict $T+2$, and so on.
  5. One-Step Projection (OSP): A direct retrieval method that attempts to jump from state $T$ to $T+k$ in a single step.

Experimental Domains

The framework was evaluated in two distinct domains:

  • The Bilestoad: A 1983 combat video game. We captured 28,000 frames of two reactive agents fighting to serve as the experience library. Each frame contained approximately 200 pieces of perceptual data.
  • Newtonian Orbit Simulation: A dataset of 20 simulated orbits used to test the mathematical accuracy of the projection methods over varying time horizons.
A frame from The Bilestoad, showing the gladiator-style combat environment used to test the agent's ability to learn from raw perceptual data.

Results and Discussion

In The Bilestoad, the PV agent achieved a performance level statistically indistinguishable from its "teacher"—a hand-coded reactive agent. This proves that projection can effectively replace complex manual rule-coding through observation.

A significant technical insight is the trade-off between PS and OSP. Projective Simulation (Step-by-Step) is better for the short term (2–11 time steps) because it can account for "cascading differences"—small changes that compound over time. However, for larger windows (22–33 steps), One-Step Projection becomes the superior method. Because OSP retrieves a real past case, the prediction is "internally consistent" and cannot "drift" into impossible states, whereas iterative simulations eventually accumulate enough error to "break out" of physical reality.

Error accumulation in the orbit domain, comparing Projective Simulation (PS) and One-Step Projection (OSP) across different time scales (TS).

Conclusion

Projective Visualization demonstrates that agents can achieve sophisticated behavior by "imagining" the future from raw experience. By prioritizing concrete data over abstract models, agents can learn complex behaviors through observation and maintain long-term consistency in their predictions. Future work will focus on enabling agents to detect where projection has broken down by comparing imagined futures against real-world outcomes.

Summary updated

BibTeX
@inproceedings{goodman1993projective,
  author = {Marc Goodman},
  title = {{Projective Visualization: Acting from Experience}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1993},
}
Figure