Using Task Descriptions in Lifelong Machine Learning for Improved Performance and Zero-Shot Transfer

David Isele, Mohammad Rostami, Eric EatonarXiv 2017 · OpenReview

Abstract

Knowledge transfer between tasks can improve the performance of learned models, but requires an accurate estimate of the inter-task relationships to identify the relevant knowledge to transfer. These inter-task relationships are typically estimated based on training data for each task, which is inefficient in lifelong learning settings where the goal is to learn each consecutive task rapidly from as little data as possible. To reduce this burden, we develop a lifelong learning method based on coupled dictionary learning that utilizes high-level task descriptions to model the inter-task relationships. We show that using task descriptors improves the performance of the learned task policies, providing both theoretical justification for the benefit and empirical demonstration of the improvement across a variety of learning problems. Given only the descriptor for a new task, the lifelong learner is also able to accurately predict a model for the new task through zero-shot learning using the coupled dictionary, eliminating the need to gather training data before addressing the task.

Introduction

In traditional Multi-Task Learning (MTL) and Lifelong Learning, an agent learns a new task by determining how it relates to previously acquired knowledge. However, most existing methods suffer from a "cold start" problem: to identify these relationships, the agent must first collect a significant amount of training data for the new task. This requirement is inefficient for real-time systems where rapid adaptation is critical.

The lifelong learning process where tasks are received consecutively and knowledge is transferred, stored, and refined.

Humans often bypass this trial-and-error phase by using high-level descriptions—for example, viewing a diagram of a new piece of furniture allows one to formulate an assembly plan before touching a tool. This paper introduces TaDeLL (Task Descriptors for Lifelong Learning), a framework that enables AI to use similar "side information" (task descriptors) to bootstrap models for new tasks instantly.

Methodology: Coupled Dictionary Learning

TaDeLL builds upon the Efficient Lifelong Learning Algorithm (ELLA) by introducing a novel Coupled Dictionary Learning approach. The system maintains two primary components:

  1. The Knowledge Base ($L$): A shared library of "knowledge chunks" (a dictionary) that are linearly combined to form the parameters ($\theta$) of a specific task.
  2. The Descriptor Dictionary ($D$): A secondary dictionary representing the space of task descriptions (e.g., physical parameters like mass or length).
Decomposition of task parameters into a shared basis and task-specific coefficients.

The innovation lies in the coupling: both dictionaries share the same sparse coefficients ($s$). This mathematical link forces the system to directly associate the description of a task with its corresponding solution.

Zero-Shot Transfer

When a new task arrives with only a description, the agent uses the descriptor dictionary ($D$) to identify the optimal coefficients ($s$). It then applies these same coefficients to the knowledge base ($L$) to instantly predict the model parameters ($\theta$). This allows for zero-shot transfer, where an effective policy can be deployed before any training data is gathered for the new task.

Experimental Evaluation

The researchers evaluated TaDeLL across three distinct machine learning paradigms:

  • Reinforcement Learning (RL): Benchmark dynamical systems including Spring-Mass Damper, Cart Pole, Bicycle balancing, and Quadrotor stabilization.
  • Regression: Predicting the 3D location of a Robot End Effector for 200 different simulated 8-DOF robot arms.
  • Classification: Synthetic domains designed to test varying sample complexities and violations of standard assumptions.

Key Results and Insights

  • Superior Performance and "Jumpstart": In RL benchmarks, TaDeLL consistently outperformed ELLA (which lacks descriptors) and standard single-task learning. It provides a massive "jumpstart," where initial performance on a new task is significantly higher than starting from scratch.
  • Computational Efficiency: TaDeLL scales linearly with the number of tasks and proved to be two orders of magnitude faster than competing task-transfer methods in runtime tests.
  • Zero-Shot Accuracy: In robot arm regression, TaDeLL’s zero-shot prediction (using zero task data) achieved a Mean Squared Error of 0.131, nearly matching the accuracy of ELLA after it had seen training data, and far exceeding single-task learners (0.73 MSE).
  • Descriptors as Regularizers: Theoretical analysis shows that task descriptors help the agent learn a more stable and distinct set of knowledge chunks, improving the quality of the underlying knowledge base.

Conclusion and Future Work

TaDeLL demonstrates that high-level task descriptions can significantly accelerate lifelong learning and enable immediate deployment through zero-shot transfer. While currently reliant on manually defined descriptors and linear coupling, future work aims to incorporate more complex descriptors, such as natural language instructions, and explore non-linear deep-learning-based couplings to handle higher-dimensional data.

Summary updated

Abstract

Knowledge transfer between tasks can improve the performance of learned models, but requires an accurate estimate of the inter-task relationships to identify the relevant knowledge to transfer. These inter-task relationships are typically estimated based on training data for each task, which is inefficient in lifelong learning settings where the goal is to learn each consecutive task rapidly from as little data as possible. To reduce this burden, we develop a lifelong learning method based on coupled dictionary learning that utilizes high-level task descriptions to model the inter-task relationships. We show that using task descriptors improves the performance of the learned task policies, providing both theoretical justification for the benefit and empirical demonstration of the improvement across a variety of learning problems. Given only the descriptor for a new task, the lifelong learner is also able to accurately predict a model for the new task through zero-shot learning using the coupled dictionary, eliminating the need to gather training data before addressing the task.
BibTeX
@misc{isele2017using,
  author = {David Isele and Mohammad Rostami and Eric Eaton},
  title = {{Using Task Descriptions in Lifelong Machine Learning for Improved Performance and Zero-Shot Transfer}},
  year = {2017},
  eprint = {1710.03850},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1710.03850},
}
Figure