Learning from failures in robot-assisted feeding: Using online learning to develop manipulation strategies for bite acquisition

Ethan K. Gordon, Xiang Meng, Matt Barnes, Tapomayukh Bhattacharjee, Siddhartha S. Srinivasa

Abstract

A successful robot-assisted feeding system requires bite acquisition of a wide variety of food items. It must adapt to changing user food preferences under uncertain visual and physical environments. Different food items in different environmental conditions require different manipulation strategies for successful bite acquisition. Therefore, a key challenge is how to handle previously unseen food items with very different success rate distributions over strategy. Combining low-level controllers and planners into discrete action trajectories, we show that the problem can be represented using a linear contextual bandit setting. We construct a simulated environment using a doubly robust loss estimate from previously seen food items, which we use to tune the parameters of off-the-shelf contextual bandit algorithms. Finally, we demonstrate empirically on a robot-assisted feeding system that, even starting with a model trained on thousands of skewering attempts on dissimilar previously seen food items, $\epsilon$-greedy and LinUCB algorithms can quickly converge to the most successful manipulation strategy.

Introduction

Robot-assisted feeding is a vital assistive technology for individuals with upper-limb impairments. However, the vast diversity of food items—each with unique physical properties like hardness, slipperiness, and geometry—presents a significant challenge. A strategy optimized for a firm carrot may fail on a soft banana slice. Current state-of-the-art systems, such as the Skewering Position Action Network (SPANet), rely on large static datasets. While effective for known items, these models often struggle with "out-of-class" foods due to covariate shift. This research explores whether a robot can learn optimal manipulation strategies for new food items on-the-fly through online learning.

Methodology: Contextual Bandits for Bite Acquisition

The researchers frame the bite acquisition task as a linear contextual bandit problem. In this framework, the robot must choose an action based on visual context and learn from the resulting success or failure.

Context and Action Space

The system uses a pre-trained RetinaNet for object detection and the penultimate layer of SPANet as a 2048-dimensional featurizer to provide the visual context ($x$). The action space ($a$) consists of six discrete skewering strategies. These are defined by combinations of fork pitch—Vertical (VS), Tilted-Vertical (TV), or Tilted-Angle (TA)—and roll orientations (parallel or perpendicular to the food's long axis).

The robot's action space includes three pitch strategies (TA, VS, TV) and two roll orientations, visualized here alongside the visual detection system.

Learning Algorithms

Two primary algorithms were evaluated:

  • $\epsilon$-greedy: A simple strategy that mostly exploits the best-known action but explores randomly with probability $\epsilon$.
  • LinUCB (Linear Upper Confidence Bound): An "optimism under uncertainty" approach that tracks the uncertainty of each action, prioritizing those with high potential or high uncertainty.

To safely tune these algorithms, the authors developed a simulator using a doubly robust estimator, allowing them to leverage historical "bandit" data from previous experiments to create a realistic offline training environment.

Results and Discussion

The core finding is that static models, despite extensive training, often fail when encountering novel food items with different physical properties.

Success rate estimates for various food items. The significant drop in performance for "Out-Of-Class" items (like bananas) underscores the limitations of static models.

Rapid Adaptation

In real-robot trials using a JACO arm, the online learning framework demonstrated rapid convergence. Even when the initial model was biased by dissimilar foods, the system typically identified the optimal strategy within approximately 10 failed attempts. LinUCB proved particularly stable, as its explicit modeling of uncertainty prevented it from prematurely discarding effective strategies after a single failure.

The "Banana" Case Study

Bananas served as a critical test case. Because they are soft and slippery, they require a specific "Tilted-Angle" (TA) approach. Models trained on firmer items (like carrots) initially failed, but the bandit algorithms quickly adapted, switching from standard vertical skewers to the successful TA strategy.

Conclusion and Future Work

This work demonstrates that online learning can effectively bridge the gap between static training datasets and the unpredictable variety of real-world food. By treating failures as informative data points, robots can become more resilient and adaptive. Future research will look toward expanding the action space into continuous variables, handling cluttered plates where items are touching, and integrating haptic feedback to better assess food properties before a skewer attempt is completed.

Summary updated

Abstract

A successful robot-assisted feeding system requires bite acquisition of a wide variety of food items. It must adapt to changing user food preferences under uncertain visual and physical environments. Different food items in different environmental conditions require different manipulation strategies for successful bite acquisition. Therefore, a key challenge is how to handle previously unseen food items with very different success rate distributions over strategy. Combining low-level controllers and planners into discrete action trajectories, we show that the problem can be represented using a linear contextual bandit setting. We construct a simulated environment using a doubly robust loss estimate from previously seen food items, which we use to tune the parameters of off-the-shelf contextual bandit algorithms. Finally, we demonstrate empirically on a robot-assisted feeding system that, even starting with a model trained on thousands of skewering attempts on dissimilar previously seen food items, $\epsilon$-greedy and LinUCB algorithms can quickly converge to the most successful manipulation strategy.
BibTeX
@inproceedings{gordon2020learning,
  author = {Ethan K. Gordon and Xiang Meng and Matt Barnes and Tapomayukh Bhattacharjee and Siddhartha S. Srinivasa},
  title = {{Learning from failures in robot-assisted feeding: Using online learning to develop manipulation strategies for bite acquisition}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=gffaeNZ_1f},
}
Figure