Introduction
Robotic pouring in unstructured environments like home kitchens remains a significant challenge due to two primary factors: transparency and carbonation. Standard computer vision systems often fail to accurately perceive transparent glassware or clear liquids. Furthermore, carbonated beverages introduce foam, which expands unpredictably, complicating real-time volume estimation.
Researchers at Carnegie Mellon University have addressed these issues by developing a zero-shot robotic pipeline that achieves high precision without requiring any task-specific training data. By combining off-the-shelf vision models with a natural language interface, the system can pour various beverages into transparent containers with less than 1% error in final liquid levels.
System Architecture
The proposed system bridges high-level human intent with low-level robotic control through a modular architecture that requires no custom annotation.
The pipeline consists of four main components:
- Natural Language Interface: A ChatGPT-powered interface translates vague user requests (e.g., "I want to drink some Coke") into specific target parameters, such as desired liquid levels.
- Vision Module: The system employs YOLO v8 for initial container detection and a Fully Convolutional Network (FCN), pre-trained on the LabPics dataset, to segment the internal state of the container. This allows the robot to distinguish between the liquid and the foam layer separately.
- Control Strategy: A PID controller manages a Franka-Emika Panda arm. The process includes an "Initial Pour" phase—tilting the source container immediately to 55 degrees to overcome surface tension—and a "wait-to-observe" policy to allow foam to settle.
- Volume Estimation: A physics-based model, assuming a cylindrical source container (like a standard soda can), maps the robot's joint rotation to the volume dispensed using degree-3 polynomial regression.
Managing Carbonation and Transparency
A key innovation of this work is the explicit segmentation of foam as a distinct state. By monitoring the ratio of foam to liquid in real-time, the controller can make intelligent decisions, such as pausing the pour to let the "head" dissipate before reaching the final target level.
The system leverages "found" models—those trained on general datasets like COCO and LabPics—to achieve robust performance in household settings. This zero-shot capability means the system works out of the box using only a standard RGB camera.
Experimental Results
The pipeline was evaluated using various beverages, including Coke, Mountain Dew, Sprite, and water, across different container geometries and camera angles.
- High Accuracy: The system consistently achieved final fluid levels within 1% of the target.
- Precision: For carbonated drinks like Coke, the standard deviation ($\sigma$) of the final level remained below 1.5%. For non-carbonated water, precision was as high as 0.2%.
- Volume Estimation: Average volume errors were remarkably low, staying below 7ml for Coke and 3ml for water, which is highly impressive given the total volume of a 355ml can.
Conclusion and Future Work
This research demonstrates that combining large-scale pre-trained vision models with LLM-based interfaces can solve complex manipulation tasks in a zero-shot manner. While highly effective, current limitations include the requirement for fixed container locations and assumptions regarding the source container's cylindrical geometry. Future work will focus on "multi-pour" tasks where the robot must find and fill multiple cups scattered in a cluttered scene and adapting to more irregular container shapes.
Summary updated