A Relational Representation for Procedural Task Knowledge

Stephen Hart, Roderic A. Grupen, David D. JensenAAAI 2005

Abstract

Autonomous robots require more than just high-level instructions; they need "procedural knowledge" to handle the nuances of physical interaction. This paper proposes a framework using Relational Dependency Networks (RDNs) to model the statistical dependencies between sensorimotor features and task success. By treating task components as relational objects, robots can autonomously identify salient environmental cues—such as object dimensions or reach orientation—necessary for successful manipulation.

Introduction

In robotics, task knowledge is typically categorized into declarative and procedural types. Declarative knowledge defines the abstract sequence of operations (e.g., "locate, then grasp"). Procedural knowledge, however, involves the specific parameters of execution (e.g., "use a three-fingered grasp because the object is wide"). While declarative structures are often provided by programmers, procedural details are highly context-dependent and difficult to pre-specify.

Traditional machine learning approaches often assume that data is independent and identically distributed (i.i.d.). In robotic manipulation, this assumption fails: the success of a grasp is intrinsically linked to the preceding reach, and the structure of trials can vary significantly. This research addresses the need for a representation that captures these complex, non-i.i.d. relationships without requiring human experts to pre-select important features.

Relational Dependency Networks for Task Knowledge

The authors utilize Relational Dependency Networks (RDNs) to represent procedural knowledge. RDNs are graphical models specifically designed for relational data, allowing the system to learn dependencies across different types of entities and actions.

Controller-Based Units

The framework decomposes tasks into "Controller-Based Units." Instead of learning from raw motor commands, the model operates on higher-level closed-loop controllers such as LOCALIZE, REACH, and GRASP. Each controller is associated with specific attributes that the RDN monitors to determine task success.

Figure 1: Task decomposition into controller-based units (Localize, Reach, Grasp) and their associated sensorimotor attributes.

Relational Modeling and Discovery

The system employs Relational Probability Trees (RPTs) to estimate the probability of an attribute's value based on other environmental or action-related variables. This allows the robot to "decorate" a known declarative schema with optimal resource allocations (e.g., choosing the number of fingers) based on real-time observations. The PROXIMITY system is used to mine these statistical correlations from experimental data.

Figure 3: The RDN framework: (a) relational data instances, (b) the abstract relational schema, and (c) the resulting unrolled dependency network.

Experimental Evaluation

The approach was validated using Dexter, a humanoid robot with two three-fingered hands. In 100 trials, Dexter attempted to pick up cylindrical cans and rectangular boxes in various orientations. The system captured visual data (bounding boxes), proprioceptive data (reach orientation), and outcome data (grasp success and lift-ability).

Key Findings

  • Automatic Salience Discovery: The RPTs successfully identified which features were critical for success. For instance, the model learned that object width and reach orientation were vital for "lift-ability," while the specific finger count became irrelevant once a stable grasp was achieved.
  • Policy Optimization: The model developed specialized policies for different objects. For a horizontal box, it learned that a top-down reach was mandatory for success, whereas for a cylindrical can, multiple reach angles were viable.
  • Affordances as Dependencies: The paper frames "affordances" as learned statistical dependencies. By discovering that "lift-ability" depends on "object width," the robot effectively learns the physical constraints and possibilities of its environment.

Conclusion and Future Work

The RDN framework provides a robust method for robots to learn procedural nuances autonomously. While the current model relies on a pre-defined declarative structure, future work aims to enable robots to learn the high-level task sequences themselves. Additionally, the authors suggest moving beyond static attribute snapshots toward dynamic "phase-space trajectories" to capture the temporal richness of robotic motion.

Summary updated

BibTeX
@inproceedings{hart2005relational,
  author = {Stephen Hart and Roderic A. Grupen and David D. Jensen},
  title = {{A Relational Representation for Procedural Task Knowledge}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2005},
}
Figure