Abstraction in Data-Sparse Task Transfer (Extended Abstract)

Tesca Fitzgerald, Ashok K. Goel, Andrea Thomaz

Introduction

Robots often struggle to transfer skills learned in controlled laboratory settings to the messy, variable real world. A robot trained to scoop pasta in one kitchen may fail if the bowl is moved, the spoon is replaced with a ladle, or the table is rotated. Traditional solutions involve massive datasets covering every possible variation, which is impractical for many applications. This research introduces Tiered Task Abstraction (TTA), a framework designed for data-sparse task transfer. TTA allows robots to generalize skills by selectively "abstracting" or forgetting specific task details, enabling them to handle structural, perceptual, and functional changes without exhaustive retraining.

Tiered Task Abstraction (TTA)

The core of the TTA framework is a mathematical decomposition of a task into a hierarchical chain:

  1. Action Model ($a$): The fundamental movement trajectory (e.g., a Dynamic Movement Primitive).
  2. Parameterization Function ($p$): Defines task-specific spatial targets (e.g., "scoop at the surface of the source container").
  3. Feature Selector ($f$): Identifies relevant environmental entities (e.g., which object is the bowl).
  4. Feature Values ($E$): Raw sensory inputs.

By moving up the abstraction hierarchy, the robot intentionally discards lower-level information, which must then be "grounded" in the new environment through perception or human input.

Tiered Task Abstraction (TTA) framework showing the relationship between retained and grounded knowledge across abstraction levels.

The framework defines three primary levels:

  • Abstraction 1: Retains the action, parameters, and feature selectors; requires only new feature values (ideal for displaced objects).
  • Abstraction 2: Retains the action and parameters; requires re-selecting features (ideal for visually new objects).
  • Abstraction 3: Retains only the action model; requires re-learning both parameters and features (necessary for functional changes like new tools).

Experimental Evaluation

The authors evaluated TTA using a 7-DOF Curi robot performing scooping and table-setting tasks across 90 transfer scenarios. These scenarios were categorized into displaced objects, replaced objects (visually different but functionally similar), and new constraints (e.g., rotated scenes or tools with different dimensions).

The Curi robot performing a scooping task in a transfer scenario.

Key Results

The experiments validated the hierarchy's effectiveness:

  • Abstraction 1 achieved 100% success for displaced objects but failed in more complex scenarios.
  • Abstraction 2 maintained high success (90-100%) for replaced objects but struggled with functional constraints.
  • Abstraction 3 proved the most robust, achieving 70-80% success in the most challenging "New Constraint" scenarios, such as using a long-handled scoop or operating on a rotated table.

Discussion and Future Work

The findings highlight a fundamental trade-off: higher levels of abstraction provide greater generality but require more "grounding" data to be executable. While Abstraction 3 can handle significant environmental shifts, it places a higher demand on the system to fill in the missing parameters.

A current limitation is that grounding information was provided manually in these experiments. Future research will focus on interactive grounding, where robots autonomously use perception or natural language queries to resolve ambiguities in novel environments. This shift toward collaborative grounding aims to make robots more adaptable and less dependent on expert intervention during deployment.

Summary updated

Abstract

When a robot adapts a learned task for a novel environment, any changes to objects in the novel environment have an unknown effect on its task execution. For example, replacing an object in a pick-and-place task affects where the robot should target its actions, but does not necessarily affect the underlying action model. In contrast, replacing a tool that the robot will use to complete a task will effectively alter its end-effector pose with respect to the robot's base coordinate system, and thus the robot's motion must be replanned accordingly. These examples highlight the relationship among (i) differences between the source and target environments, (ii) the level of abstraction at which a robot's task model should be represented to enable transfer to the target environment, and (iii) the information needed to ground the abstracted task representation in the target environment. In this abstract, summarizing our full article [Fitzgerald et al., 2021], we present our taxonomy of transfer problems based on this relationship. We also describe a knowledge representation called the Tiered Task Abstraction (TTA) and demonstrate its applicability to a variety of transfer problems in the taxonomy. Our experimental results indicate a trade-off between the generality and data requirements of a task representation, and reinforce the need for multiple transfer methods that operate at different levels of abstraction.
BibTeX
@inproceedings{fitzgerald2022abstraction,
  author = {Tesca Fitzgerald and Ashok K. Goel and Andrea Thomaz},
  title = {{Abstraction in Data-Sparse Task Transfer (Extended Abstract)}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/800},
}

Loading related content...

Figure