FoAM: Foresight-Augmented Multi-Task Imitation Policy for Robotic Manipulation

Litao Liu, Wentao Wang, Yifan Han, Zhuoli Xie, Pengfei Yi, Junyan Li, Qin Yi, Wenzhao Lian

Abstract

Multi-task imitation learning (MTIL) has shown significant potential in robotic manipulation by enabling agents to perform various tasks using a single policy. This simplifies policy deployment and enhances adaptability. However, challenges remain in maintaining action reliability and generalizing to unseen tasks. We introduce the Foresight-Augmented Manipulation Policy (FoAM), a novel MTIL policy that utilizes multi-modal goal conditions and introduces foresight augmentation alongside general action reconstruction. FoAM enables agents to reason about the visual consequences of their actions, learning expressive embeddings that capture nuanced task variations. Extensive experiments across 100+ tasks in simulation and real-world settings demonstrate that FoAM outperforms state-of-the-art baselines by up to 41% in success rate.

Introduction

In robotic manipulation, the goal of Multi-Task Imitation Learning (MTIL) is to develop a single policy capable of executing a wide array of chores. Despite its promise, MTIL faces two primary hurdles: action reliability and scenario ambiguity. Robots often fail to reason about the long-term consequences of their movements, leading to drifts from intended trajectories. Furthermore, language-only models struggle with generalization, while image-only models often fail to resolve ambiguities in complex environments, such as identifying which specific drawer in a cabinet is the target.

FoAM addresses these issues by equipping robots with "foresight"—the human-like ability to visualize the completed state of a task and use that mental image to guide real-time actions.

Methodology

FoAM introduces two primary innovations to the standard Transformer-based policy architecture to enhance task understanding and execution reliability.

Multi-modal Goal Conditioning

Rather than relying solely on text or images, FoAM utilizes both. A Goal Imagination Module, powered by a fine-tuned Vision-Language Model (InstructPix2Pix), takes a text command (e.g., "Put eggplant into the green bowl") and the current camera view to "hallucinate" a visual representation of the successful goal state.

Foresight Augmentation (FA)

During training, the model is tasked with predicting not only the next sequence of actions but also the visual consequence of those actions. A Foresight Loss ensures the robot's predicted future state aligns with the imagined goal image. This forces the model to learn a deeper understanding of how its movements alter the environment, acting as a powerful regularizer that prevents "nonsense" actions.

FoAM Architecture: The model integrates a Goal Imagination Module and Foresight Augmentation to predict visual consequences of actions.

Experimental Setup

The researchers evaluated FoAM using a massive new benchmark and real-world tests:

  • FoAM Benchmark: A MuJoCo-based simulation suite featuring a dual-arm UR3e robot with 10 scenarios and 86 tasks.
  • Real-World Dataset: 14 tasks (e.g., test tube sorting, fruit placing) using a UFACTORY xArm 7.
  • VLM Training: The imagination module was trained on 20,000 image pairs, including data from Google's RT-1 dataset.

Key Results and Discussion

FoAM demonstrated significant performance gains across all tested scenarios. In simulation, the model achieved an average success rate of 86% in dual-arm tasks, vastly outperforming baselines like BAKU (31%) and MT-ACT (33%).

Comparative performance of FoAM against state-of-the-art baselines across various simulation task categories.

Generalization and Robustness

On completely unseen tasks, FoAM maintained a 66% success rate, whereas language-based baselines often dropped to 0%. This suggests that aligning pixel-level information with semantic instructions allows for zero-shot potential in novel scenarios. In physical tests, FoAM proved reactive; for instance, if an object was removed from its gripper, the robot could re-grasp it and continue the task rather than blindly following a pre-planned path.

Limitations and Future Work

While FoAM sets a new standard for MTIL, some limitations persist. Tasks requiring extreme precision, such as inserting a test tube into a tight hole, sometimes suffered from slight misalignments in the VLM-generated goal images. Future research may focus on generating intermediate sub-goals to act as consequence controllers throughout long-horizon tasks, further refining the robot's ability to navigate complex, multi-step environments.

Summary updated

Abstract

Multi-task imitation learning (MTIL) has shown significant potential in robotic manipulation by enabling agents to perform various tasks using a single policy. This simplifies the policy deployment and enhances the agent's adaptability across different scenarios. However, key challenges remain, such as maintaining action reliability (e.g., avoiding abnormal action sequences that deviate from nominal task trajectories) and generalizing to unseen tasks with a few expert demonstrations. To address these challenges, we introduce the Foresight-Augmented Manipulation Policy (FoAM), a novel MTIL policy that pioneers the use of multi-modal goal condition as input and introduces a foresight augmentation in addition to the general action reconstruction. FoAM enables the agent to reason about the visual consequences (states) of its actions and learn more expressive embedding that captures nuanced task variations. Extensive experiments on over 100 tasks in simulation and real-world settings demonstrate that FoAM significantly enhances MTIL policy performance, outperforming state-of-the-art baselines by up to 41% in success rate. Meanwhile, we released our simulation suites, including a total of 10 scenarios and over 80 challenging tasks designed for manipulation policy training and evaluation. See the project homepage projFoAM.github.io for project details.
BibTeX
@misc{liu2024foam,
  author = {Litao Liu and Wentao Wang and Yifan Han and Zhuoli Xie and Pengfei Yi and Junyan Li and Qin Yi and Wenzhao Lian},
  title = {{FoAM: Foresight-Augmented Multi-Task Imitation Policy for Robotic Manipulation}},
  year = {2024},
  eprint = {2409.19528},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.19528},
}
Figure