TL;DR
This paper explores using Vision-Language Models (VLMs) like MiniGPT-4 to automatically detect if a robot has successfully completed a task, framing it as a Visual Question Answering (VQA) problem. They demonstrate that fine-tuning a VLM on a small amount of robot data can create a highly accurate success detector, and importantly, that these models are much more robust to variations in task descriptions than to visual changes.
What Problem Does This Paper Address?
For robots to operate autonomously and learn effectively, they need to understand when they've achieved their goals. Manually annotating robot task success is time-consuming and doesn't scale. This paper tackles the challenge of creating an automated "success detector" that can reliably tell if a robot has completed a task, even when the robot's observations (images, sensor data) or the task descriptions vary. This is crucial for enabling robots to learn from experience, recover from errors, and improve their performance through reinforcement learning, where success signals can act as sparse rewards.
Methods
The core idea is to leverage the power of pre-trained Vision-Language Models (VLMs) for this success detection task. Specifically, the authors use MiniGPT-4, an open-source VLM that aligns a frozen visual encoder with a frozen large language model. They frame the problem as a Visual Question Answering (VQA) task: given a robot's observation (e.g., an image of the robot's workspace) and a textual description of the task, the VLM answers a question like "Is the cloth swept to the left side of the table?" with a simple "Yes" or "No."
The key methodological contribution is the fine-tuning of MiniGPT-4 on robot trajectory data. They explore how much data is needed for effective fine-tuning (few-shot learning) and investigate the impact of different training strategies, such as co-finetuning on multiple datasets. They also rigorously test the model's robustness to variations in both the visual input (e.g., different viewpoints, lighting) and the language used to describe the task.
Datasets
The paper utilizes two primary datasets:
- Berkeley Bridge Dataset: A large dataset containing 60,096 trajectories across 13 skills and 24 diverse environments (kitchens, tabletops, sinks, laundry machines).
- Berkeley UR5 Demonstration Dataset: A smaller dataset with 896 trajectories and 4 tasks, all set in a tabletop environment.
These datasets provide real-world robot demonstrations for training and evaluating the success detector.
Key Insights and Ideas
- Few-Shot Learning is Powerful: A surprisingly small number of robot trajectories (as few as 5) are sufficient to fine-tune MiniGPT-4 into a highly accurate success detector for specific tasks. This significantly reduces the data annotation burden.
- Language Robustness Outperforms Visual Robustness: The VLM-based success detector is remarkably resilient to variations in how a task is described (e.g., using synonyms, rephrasing). However, it is more sensitive to visual changes like different viewpoints, lighting, or added objects. This suggests that while VLMs are good at understanding language, the visual perception component might be the bottleneck for generalization in complex robotic environments.
- Distribution Shift is a Challenge: Transferring a success detector trained on one robot dataset (e.g., Bridge) to another with different environments or tasks can lead to a significant drop in performance. This highlights the difficulty of achieving true generalizability across diverse robotic scenarios.
- Zero-Shot Performance is Limited: Without fine-tuning, the model's zero-shot accuracy is poor, indicating that task-specific adaptation is necessary for reliable success detection.
Key Results
- High Accuracy on Specific Tasks: After fine-tuning with as few as 5 trajectories on UR5 tasks, balanced accuracy reached 92% and 94% for cloth sweeping and cup stacking, respectively.
- Generalization to Unseen Tasks (Limited): When fine-tuned on the Bridge dataset and tested on unseen tasks within that dataset, MiniGPT-4 achieved a test accuracy of 70.3%. However, it struggled to generalize to unseen UR5 tasks after being trained on UR5 data.
- Robustness Findings: Balanced accuracy dropped by 20% with visual augmentations but only by 1% with language paraphrasing, confirming the greater sensitivity to visual variations.
Limitations and Future Work
The authors acknowledge that their success detector struggles with significant distribution shifts between datasets, limiting its generalizability to entirely new environments or tasks without further fine-tuning. They also note that while MiniGPT-4 is powerful, more advanced VLMs like GPT-4 and Gemini might offer even better performance, especially in zero-shot settings.
Future work directions include:
- Exploring the potential of more powerful VLMs (GPT-4, Gemini) for zero-shot success detection.
- Investigating how success detection can be effectively used as a sparse reward signal to improve existing robot learning policies.
- Developing methods to improve the visual robustness of these detectors to handle greater environmental variations.
Summary updated