A learning of object structures by verbalism

Norihiro Abe, Saburo TsuJlCOLING 1982

TL;DR

This paper presents a system that learns object structures through verbal instructions, augmenting its existing knowledge base when encountering unknown objects. It tackles the challenge of object recognition by allowing explicit teaching of new descriptions, bridging the gap between pre-defined models and real-world object variations.

What Problem Does This Paper Address?

Traditional AI systems for object recognition rely on pre-defined models. This approach faces significant limitations: it's difficult to store models for every possible object variation, and it struggles to discriminate subtle differences between similar objects (e.g., different types of houses). When an unknown object appears, or when an object doesn't perfectly match an existing model, the system falters. This paper addresses the need for a more flexible learning mechanism where a system can acquire new knowledge about object structures through explicit, verbal instruction, much like a human learns from a teacher.

Methods

The core of the system's learning capability lies in its "verbalism" approach, where it uses both linguistic and pictorial information. When faced with an unknown object or a discrepancy, the system can be explicitly taught new descriptions. This teaching process involves:

  • Frame Representation: Objects and their relationships are represented using a hierarchical frame structure. Slots like AKO (a kind of), CLASS (generic/instance), PART, POS (position), DIR (direction), SHAP (shape), and SUBP (subparts) are used to describe object attributes and their interconnections.
  • Analogy and Discrepancy Detection: The system attempts to find analogies with existing models based on subpart locations. When discrepancies arise, it relies on verbal feedback from a "teacher" to understand and correct its interpretations.
  • Explanation Capability: A crucial component is the system's ability to explain its current understanding and its uncertainties to the teacher. This allows the teacher to provide targeted instructions. The system can explain its reasoning based on pictorial features and relational constraints.

Datasets

Not discussed in the paper. The paper uses a "sample story" and figures (Fig. 1, 2, 4, 5) to illustrate its concepts, but no formal datasets are mentioned for training or evaluation.

Key Insights and Ideas

  • Explicit Teaching is Crucial for Robust Object Learning: The paper strongly advocates for explicit verbal instruction as a primary mechanism for learning about unknown objects and their variations, moving beyond purely automatic generalization.
  • Discrepancy-Driven Learning: The system actively seeks out and highlights discrepancies between its current model and observed objects, using these as prompts for teacher intervention.
  • Hierarchical Frame Structures Enable Detailed Object Description: The use of frames with slots for parts, positions, shapes, and relationships allows for a rich and structured representation of object knowledge.
  • Explanation as a Learning Enabler: The system's ability to explain its current state of knowledge is as important as its ability to learn, as it guides the teacher in providing effective instructions.

Key Results

The paper demonstrates the system's ability to learn about a new character, OJIRO, by augmenting an existing model (OBAQ) and creating a new instance. Through a simulated dialogue with a teacher, the system successfully identifies OJIRO's attributes, such as its single hair and yellow body color, by incorporating verbal corrections. The example scenario shows the system progressively refining its understanding of OJIRO's mouth and eyes based on teacher feedback, ultimately leading to a successful identification.

Limitations and Future Work

  • Reliance on Teacher: The system's learning is heavily dependent on the teacher's ability to provide clear and accurate instructions. The paper acknowledges that the system currently cannot fully grasp the teacher's intentions or requirements, leading to potential ambiguities in explanations.
  • Ambiguity Resolution: While the system can identify discrepancies, it sometimes struggles to resolve ambiguities, especially when multiple parts could be candidates for a given feature.
  • Generalization of Generic Frames: The paper notes that the generation of generic frames (like BOGLE from OBAQ) is still an area needing more consideration, particularly in compiling simple pictorial descriptions from various deviations.
  • Future Work: The authors suggest that future work should focus on improving the system's ability to grasp the teacher's intention and requirements to provide simpler and more effective explanations. They also highlight the need for better strategies to discover cues for finding analogies between subparts, as current methods might not be universally applicable.

Summary updated

BibTeX
@inproceedings{abe1982learning,
  author = {Norihiro Abe and Saburo TsuJl},
  title = {{A learning of object structures by verbalism}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1982},
}
Figure