Qualitatively Describing Objects Using Spatial Prepositions

Alicia Abella, John R. KenderAAAI 1993

Abstract

This paper presents a computational framework for translating low-level geometric data into high-level qualitative spatial descriptions, such as "near," "far," or "next to." By abstracting complex object shapes into "blobs" defined by statistical moments and applying fuzzy set theory to geometric inequalities, the authors develop a system capable of handling the inherent vagueness of human language. The approach allows machines to interpret spatial relationships not as binary states, but as degrees of membership, facilitating more natural human-computer interaction.

Introduction

While computers excel at processing precise numerical coordinates, human communication relies on vague, context-dependent spatial prepositions. Bridging the gap between low-level computer vision (pixels and moments) and high-level qualitative description remains a significant challenge. This research addresses the problem of mathematically defining prepositions so that a machine can describe scenes in a way that feels natural to a human observer, without relying on fixed reference objects.

Methodology

The proposed pipeline transforms raw geometry into linguistic labels through several stages:

  1. Object Representation: Objects are simplified into "blobs" represented by a 6-element vector derived from 0th, 1st, and 2nd order statistical moments. This captures essential features like area, center of mass, and the inertia tensor (representing "stretch").
  2. Geometric Definitions: Spatial prepositions are defined as regions within a 12-dimensional space (formed by pairs of 6D object vectors) governed by specific inequalities. For instance, "Near" is defined by the intersection of bounding boxes, while "Aligned" is determined by the principal axes of inertia.
  3. Fuzzification and Monte Carlo Simulation: To account for linguistic vagueness, the system uses Fuzzy Set Theory to calculate a membership grade between 0 and 1. Because the 12D inequalities are complex, a Monte Carlo approach is used: random points are scattered around object coordinates, and the ratio of points satisfying the "ideal" preposition boundaries determines the fuzzy membership value.

Experimental Evaluation

The system was tested using a custom image containing seven irregular "blob" shapes designed to represent various spatial configurations.

Figure 4: The experimental test image featuring seven irregular shapes used to evaluate spatial preposition identification.

The system successfully identified intuitive relationships. For example, objects 1 and 3 were correctly labeled as "next to" each other. The fuzzy logic component was particularly effective for borderline cases; when testing objects 2 and 6 for alignment, a strict geometric test failed, but the fuzzy system returned a high membership grade (0.8), aligning with human perception.

Key Insights: Resolving Ambiguity

A core contribution of this work is the use of fuzzy membership curves to resolve ambiguity between competing prepositions. By analyzing the slope of these curves, the system can distinguish between "somewhat near" and "somewhat far" based on which membership value changes more rapidly as boundaries are "fuzzed."

Figure 6: Fuzzy membership curves for 'far' and 'near' prepositions as a function of the vagueness parameter σ.

As shown in Figure 6, the system can tune a "vagueness" threshold ($\sigma$) to adjust how strictly it applies labels. This allows the model to mimic human flexibility in description, where a relationship might be described as "mostly aligned" rather than a binary "true" or "false."

Conclusion and Future Work

The "blob" abstraction effectively simplifies spatial reasoning without losing the essence of object relationships. While currently limited to 2D space and ignoring broader context (like object size or purpose), the framework provides a robust foundation for qualitative spatial reasoning. Future work suggests extending the model to 3D and incorporating interactive dialogue systems to refine descriptions through human feedback.

Summary updated

BibTeX
@inproceedings{abella1993qualitatively,
  author = {Alicia Abella and John R. Kender},
  title = {{Qualitatively Describing Objects Using Spatial Prepositions}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1993},
}
Figure