Computer Interpretation of Human Stick Figures

Martin HermanAAAI 1980

Abstract

In the early 1980s, computer vision research primarily focused on low-level object identification. Martin Herman proposed a shift toward high-level interpretation, viewing vision as a medium for understanding human action and intent. This paper explores SKELETUN, a system that utilizes 2D stick figures to bridge the gap between raw spatial data and semantic meaning, such as identifying sports activities or emotional states.

Introduction

While contemporary vision systems struggled with the computational complexity of processing full human forms, SKELETUN bypassed these issues by using stick figures. These abstractions preserve the "gross body posture" essential for communicating body language while removing the noise of textures and clothing. The goal was to answer not just "what" is in an image, but "what are these people doing, and why?"

Methods: The Three-Tiered Hierarchy

SKELETUN operates through a structured hierarchy that transforms simple geometric data into conceptual interpretations.

1. 2D Description

The process begins with a low-level encoding of line segments, joint angles, and overlapping parts. Input consists of hand-encoded x, y coordinates for segment endpoints and the head's center.

Figure 1: A stick figure with labeled segments used for 2D description.

2. Physical Space Description

The system then generates a 3D, object-centered interpretation. Rather than using global coordinates, it describes positions relative to the torso (e.g., the elbow's position relative to the shoulder). Predicates like OUT, IN, or IN-MIDDLE are used to define these spatial relationships.

3. Meaning Space Description

The final layer involves conceptual interpretation using heuristic inference rules. For instance, to identify "fielding a grounder," the system evaluates if the figure is vertical, if the lower body is crouching, and if the arms are positioned for the task. SKELETUN also employs discrete confidence values, allowing it to suggest multiple possible interpretations with varying degrees of certainty.

Results and Domain Application

The system was tested across two primary domains: baseball and emotions.

Figure 2: Examples of upright stick figures representing different poses.

In the baseball domain, SKELETUN successfully distinguished between catching a ball at chest height and fielding a grounder by analyzing knee bends and torso orientation. In the emotional domain, it categorized figures into states like "happy" or "sad" based on posture. The system demonstrated robustness by handling diagonal orientations and adjusting physical inference rules for leaning figures.

Discussion and Limitations

SKELETUN provided a qualitative proof-of-concept for semantic reasoning from abstract geometry. However, several limitations remain. The system relies on static images and cannot interpret temporal sequences or motion. Additionally, it requires manual coordinate input and cannot handle extreme orientations, such as upside-down figures. Future work would involve automating coordinate extraction and expanding the "Meaning Space" to broader domains.

Conclusion

SKELETUN highlights the power of abstraction in computer vision. By focusing on relative hierarchies and context-sensitive interpretation, it demonstrated that high-level semantic understanding is possible even with minimal visual detail.

Summary updated

BibTeX
@inproceedings{herman1980computer,
  author = {Martin Herman},
  title = {{Computer Interpretation of Human Stick Figures}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1980},
}
Figure