Pour me a drink: Robotic Precision Pouring Carbonated Beverages into Transparent Containers

Feiya Zhu, Shuo Hu, Letian Leng, Alison Bartsch, Abraham George, Amir Barati FarimaniarXiv 2023 · DOI · arXiv

Introduction

Robotic pouring in unstructured environments like home kitchens remains a significant challenge due to two primary factors: transparency and carbonation. Standard computer vision systems often fail to accurately perceive transparent glassware or clear liquids. Furthermore, carbonated beverages introduce foam, which expands unpredictably, complicating real-time volume estimation.

Researchers at Carnegie Mellon University have addressed these issues by developing a zero-shot robotic pipeline that achieves high precision without requiring any task-specific training data. By combining off-the-shelf vision models with a natural language interface, the system can pour various beverages into transparent containers with less than 1% error in final liquid levels.

System Architecture

The proposed system bridges high-level human intent with low-level robotic control through a modular architecture that requires no custom annotation.

The modular pipeline connects user input via ChatGPT to vision-based control and robotic execution.

The pipeline consists of four main components:

  1. Natural Language Interface: A ChatGPT-powered interface translates vague user requests (e.g., "I want to drink some Coke") into specific target parameters, such as desired liquid levels.
  2. Vision Module: The system employs YOLO v8 for initial container detection and a Fully Convolutional Network (FCN), pre-trained on the LabPics dataset, to segment the internal state of the container. This allows the robot to distinguish between the liquid and the foam layer separately.
  3. Control Strategy: A PID controller manages a Franka-Emika Panda arm. The process includes an "Initial Pour" phase—tilting the source container immediately to 55 degrees to overcome surface tension—and a "wait-to-observe" policy to allow foam to settle.
  4. Volume Estimation: A physics-based model, assuming a cylindrical source container (like a standard soda can), maps the robot's joint rotation to the volume dispensed using degree-3 polynomial regression.

Managing Carbonation and Transparency

A key innovation of this work is the explicit segmentation of foam as a distinct state. By monitoring the ratio of foam to liquid in real-time, the controller can make intelligent decisions, such as pausing the pour to let the "head" dissipate before reaching the final target level.

Time-series analysis of the pouring process, illustrating the dynamic segmentation of foam (red) and liquid (green) levels over time.

The system leverages "found" models—those trained on general datasets like COCO and LabPics—to achieve robust performance in household settings. This zero-shot capability means the system works out of the box using only a standard RGB camera.

Experimental Results

The pipeline was evaluated using various beverages, including Coke, Mountain Dew, Sprite, and water, across different container geometries and camera angles.

  • High Accuracy: The system consistently achieved final fluid levels within 1% of the target.
  • Precision: For carbonated drinks like Coke, the standard deviation ($\sigma$) of the final level remained below 1.5%. For non-carbonated water, precision was as high as 0.2%.
  • Volume Estimation: Average volume errors were remarkably low, staying below 7ml for Coke and 3ml for water, which is highly impressive given the total volume of a 355ml can.

Conclusion and Future Work

This research demonstrates that combining large-scale pre-trained vision models with LLM-based interfaces can solve complex manipulation tasks in a zero-shot manner. While highly effective, current limitations include the requirement for fixed container locations and assumptions regarding the source container's cylindrical geometry. Future work will focus on "multi-pour" tasks where the robot must find and fill multiple cups scattered in a cluttered scene and adapting to more irregular container shapes.

Summary updated

Abstract

With the growing emphasis on the development and integration of service robots within household environments, we will need to endow robots with the ability to reliably pour a variety of liquids. However, liquid handling and pouring is a challenging task due to the complex dynamics and varying properties of different liquids, the exacting precision required to prevent spills and ensure accurate pouring, and the necessity for robots to adapt seamlessly to a multitude of containers in real-world scenarios. In response to these challenges, we propose a novel autonomous robotics pipeline that empowers robots to execute precision pouring tasks, encompassing both carbonated and non-carbonated liquids, as well as opaque and transparent liquids, into a variety of transparent containers. Our proposed approach maximizes the potential of RGB input alone, achieving zero-shot capability by harnessing existing pre-trained vision segmentation models. This eliminates the need for additional data collection, manual image annotations, or extensive training. Furthermore, our work integrates ChatGPT, facilitating seamless interaction between individuals without prior expertise in robotics and our pouring pipeline, this integration enables users to effortlessly request and execute pouring actions. Our experiments demonstrate the pipeline's capability to successfully pour a diverse range of carbonated and non-carbonated beverages into containers of varying sizes, relying solely on visual input.
BibTeX
@misc{zhu2023pour,
  author = {Feiya Zhu and Shuo Hu and Letian Leng and Alison Bartsch and Abraham George and Amir Barati Farimani},
  title = {{Pour me a drink: Robotic Precision Pouring Carbonated Beverages into Transparent Containers}},
  year = {2023},
  eprint = {2309.08892},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.08892},
}
Figure