Learning Human Utility from Video Demonstrations for Deductive Planning in Robotics

Nishant Shukla, Yunzhong He, Frank Chen, Song-Chun ZhuCoRL 2017 · OpenReview

Abstract

We uncouple three components of autonomous behavior (utilitarian value, causal reasoning, and fine motion control) to design an interpretable model of tasks from video demonstrations. Utilitarian value is learned from aggregating human preferences to understand the implicit goal of a task, explaining why an action sequence was performed. Causal reasoning is seeded from observations and grows from robot experiences to explain how to deductively accomplish sub-goals. And lastly, fine motion control describes what actuators to move. In our experiments, a robot learns how to fold t-shirts from visual demonstrations, and proposes a plan (by answering why, how, and what) when folding never-before-seen articles of clothing.

Introduction

Traditional "Learning from Demonstration" (LfD) systems often rely on inductive learning—mimicking a sequence of observed actions to replicate a task. While effective for repetitive actions, this approach is brittle; if the environment or the object changes (e.g., a long-sleeved shirt instead of a t-shirt), the robot often fails because it lacks an understanding of the underlying goal.

Researchers at UCLA propose a shift toward deductive planning. Instead of copying steps, the robot learns a "utility landscape"—a mathematical representation of which world states are most desirable. By understanding the why behind a task, the robot can use its own causal knowledge to reach high-utility states regardless of the specific human demonstration.

The Three-Layer Framework

The researchers decomposed autonomous behavior into three distinct, uncoupled layers to improve generalization and interpretability:

1. Utilitarian Value (The "Why")

The robot models human preferences as a latent utility function. It identifies "fluents"—measurable features like symmetry, height, or width—to explain why one state is better than another. This creates a value landscape where peaks represent optimal task completion.

The utility landscape learned from demonstrations. The 3D surface maps states to values, with peaks representing optimal folding configurations.

2. Causal Reasoning (The "How")

To understand how to transition between states, the system uses a Stochastic Context-Free And-Or Graph (STC-AOG). This structure represents the causal relationships between actions and their effects on fluents, allowing the robot to plan a sequence of sub-goals to reach the desired utility peak.

3. Fine Motion Control (The "What")

The bridge between high-level planning and low-level actuation is managed via "Fluent Dynamics." This control formulation uses the chain rule to map changes in robot joint angles to changes in the overall utility, ensuring that every movement contributes to the goal.

Learning Utility via Ranking Pursuit

To determine which fluents are most relevant to a task, the researchers employed Ranking Pursuit. This method greedily selects features from a dictionary that best explain the rankings observed in human demonstrations and surveys. In the folding task, features like height, width, and y-symmetry were found to be the most significant indicators of a successful fold.

Ranking Pursuit results showing the number of violations for different fluents. The system identifies height, width, and symmetry as primary indicators of a "good" fold.

Experimental Results and Generalization

The framework was evaluated using 45 RGB-D video demonstrations of t-shirt folding and a human preference survey involving 330 participants.

Key findings include:

  • Rapid Convergence: The model's internal ranking began to align with human preferences after observing as few as five demonstrations.
  • Cross-Category Generalization: Despite being trained exclusively on t-shirts, the robot successfully planned and executed folding tasks for towels, shorts, and pants. This demonstrates the power of uncoupling utility from specific object dynamics.
  • Interpretability and Failure Detection: Because the utility is based on human-understandable fluents, the robot can explain its reasoning. Furthermore, by comparing predicted utility gains against actual outcomes, the robot can autonomously detect execution failures, such as a garment slipping from its grip.

Conclusion

By uncoupling the why, how, and what of a task, this research provides a path toward more robust and interpretable robotic planning. Future work aims to automate the discovery of causal relationships through autonomous exploration and to incorporate social choice theory to handle diverse human preferences in task execution.

Summary updated

Abstract

We uncouple three components of autonomous behavior (utilitarian value, causal reasoning, and fine motion control) to design an interpretable model of tasks from video demonstrations. Utilitarian value is learned from aggregating human preferences to understand the implicit goal of a task, explaining \textitwhy an action sequence was performed. Causal reasoning is seeded from observations and grows from robot experiences to explain \textithow to deductively accomplish sub-goals. And lastly, fine motion control describes \textitwhat actuators to move. In our experiments, a robot learns how to fold t-shirts from visual demonstrations, and proposes a plan (by answering \textitwhy, \textithow, and \textitwhat) when folding never-before-seen articles of clothing.
BibTeX
@inproceedings{shukla2017learning,
  author = {Nishant Shukla and Yunzhong He and Frank Chen and Song-Chun Zhu},
  title = {{Learning Human Utility from Video Demonstrations for Deductive Planning in Robotics}},
  booktitle = {Conference on Robot Learning},
  year = {2017},
  url = {https://openreview.net/forum?id=khBNolzzZL},
}
Figure