Interpreting Prepositions Physically

Jugal K. Kalita, Norman I. BadlerAAAI 1991

Abstract

This paper presents a framework for translating natural language prepositions (like on, in, behind, and across) into precise geometric and kinematic constraints. By decomposing these spatial terms into implementable semantic primitives, the authors enable virtual "human" agents to autonomously interpret and execute physical tasks in a 3D environment.

Introduction

While linguists have long studied the nuances of spatial prepositions, their descriptions are often too abstract for computational use. For a robot or a virtual agent to "put the block on the table," it needs more than a dictionary definition; it needs to know which surfaces must touch, how the objects should be oriented, and what physical constraints (like gravity and support) must be satisfied. This research addresses the "implementability gap"—the challenge of turning ambiguous natural language into the concrete geometric calculations required for computer animation and robotics.

Methodology

The authors propose a representation template based on geometric-relation frames. Their approach breaks down prepositions into several components:

  • Positional and Orientational Constraints: Prepositions are mapped to 3D regions of space or axial alignments.
  • Semantic Primitives: Functions like self-supporting-spaces-of(X) identify parts of an object capable of providing or receiving physical support.
  • Zone Construction: For "projective" prepositions like behind, the system geometrically constructs a 3D "back zone" by drawing tangent planes to the landmark object relative to a global coordinate system.
Geometric construction of a 'back zone' for projective prepositions.
  • Path Specification: For dynamic prepositions like across, the system defines a path using source, destination, and path geometry.
  • Integration Pipeline: A bottom-up parser creates a logical representation, which is filled with geometric constraints and passed to a task planner and the JACK toolkit simulator for final animation.

Key Insights

The paper treats prepositions as geometric programs—sets of instructions for a constraint-satisfaction engine. For example, to understand "on," the system must actively search for horizontal surfaces and calculate normals.

Furthermore, the authors highlight the dual nature of "across", modeling it as both a location and a dynamic journey between specific exterior edges. To handle the complexity of real-world objects, the system uses functional labeling (e.g., "is-a-container") to resolve selectional restrictions that are difficult to derive from raw vertices alone. Finally, rather than using simple vectors, the authors define projective space using 3D volumes (zones), allowing the agent to handle objects of varying sizes and shapes more robustly.

Results and Evaluation

As a 1991 study focused on symbolic AI, the evaluation is based on representative imperatives tested within a simulated 3D environment. The primary result is a qualitative demonstration of a "human" agent performing tasks based on natural language input.

Sequence of a virtual agent executing the command 'Put the block on the table'.

The system successfully interpreted commands like "Put the block on the table" by identifying support surfaces and generating collision-free animations. The framework proved sufficient to handle complex spatial relations, including three-object relations like between, by intersecting the projection zones of two landmark objects.

Discussion and Future Work

Despite its successes, the model faces the "Odd Context" problem, where capturing all selectional restrictions (e.g., the orientation of a bowl for it to contain bread) is difficult. The current model also assumes a "canonical encounter" perspective and relies on pre-defined heuristics for identifying functional parts of objects. Future work suggests that more detailed linguistic studies could be formalized into this implementable format to cover a wider range of human spatial reasoning.

Summary updated

BibTeX
@inproceedings{kalita1991interpreting,
  author = {Jugal K. Kalita and Norman I. Badler},
  title = {{Interpreting Prepositions Physically}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1991},
}
Figure