Robotic Table Wiping via Reinforcement Learning and Whole-body Trajectory Optimization

Thomas Lew, Sumeet Singh, Mario Prats, Jeffrey Bingham, Jonathan Weisz, Benjie Holson, Xiaohan Zhang, Vikas Sindhwani, Yao Lu, Fei Xia, Peng Xu, Tingnan Zhang, Jie Tan, Montserrat Gonzalez

Introduction

Autonomous table wiping presents a significant "visuo-motor" challenge for robotics. Unlike humans, robots struggle with the unpredictable dynamics of crumbs and spills, which may stick to the wiper or be pushed around. Furthermore, a mobile manipulator must coordinate its base and arm movements—known as whole-body control—to reach across surfaces while avoiding obstacles like chairs and maintaining consistent contact force.

This paper introduces a hybrid framework that addresses these challenges by combining Reinforcement Learning (RL) for high-level strategic planning with Whole-body Trajectory Optimization for safe, constrained execution. By modeling messy dynamics using Stochastic Differential Equations (SDEs), the researchers trained a policy entirely in simulation that transfers "zero-shot" to real-world hardware.

Methodology

The proposed system decouples the cleaning task into a four-step pipeline, balancing intelligent planning with safety guarantees.

The four-step pipeline: Perception, High-level planning (RL), Trajectory optimization, and Controller.
  1. Perception: LiDAR and camera data are processed to create a 64x64 binary image mask of "dirt" and bounding boxes for obstacles. This abstraction bypasses the visual sim-to-real gap.
  2. High-Level Planning (RL): A Soft Actor-Critic (SAC) policy analyzes the mask to select the optimal "wipe" primitive, defined by a start position, orientation, and length.
  3. Stochastic Modeling: To train the RL policy without real-world data, the authors developed a simulator using Stochastic Differential Equations (SDEs). This models particle movement (Brownian motion) and the probability of dirt being cleaned (Poisson process).
  4. Whole-body Trajectory Optimization: A "ShootingSQP" optimizer calculates joint trajectories for the robot's base and arm. It ensures collision avoidance and maintains a steady 10N force on the table via admittance control.

The RL policy was trained on 50 million environment steps in simulation and validated through 1,000 simulated rollouts before real-world testing on the Everyday Robots mobile manipulator platform.

Results and Key Insights

The hybrid approach demonstrated several advantages over traditional heuristic methods. The RL-based policy cleaned surfaces significantly faster by intelligently targeting areas of high dirt density.

Real-world execution showing the robot performing wiping trajectories on a table with obstacles.

Key findings include:

  • Zero-Shot Transfer: The system successfully transitioned from simulation to real hardware without any fine-tuning.
  • Robustness to Clutter: The robot successfully gathered crumbs and wiped spills while navigating around chairs that restricted its movement.
  • Generalization: Despite being trained on simple dirt distributions, the policy effectively handled complex, multi-area spills in real-world environments.

By decoupling strategy from safety, the framework leverages the strengths of RL for "messy" logic while relying on trajectory optimization for provable safety and physical constraints.

Limitations and Future Work

While effective, the current system has limitations. The perception module relies on basic color filtering, which may fail in complex lighting or with low-contrast spills. Additionally, the robot is currently limited to straight-line wiping primitives.

Future research could explore:

  • Curved Wiping: Developing more complex, non-linear wiping motions.
  • Material Awareness: Learning SDE parameters (e.g., stickiness or absorbency) online by observing how spills react to initial contact.
  • Kinematic Generalization: Adapting the trajectory optimization for different robotic platforms beyond the specific kinematics used in this study.

Summary updated

Abstract

We propose a framework to enable multipurpose assistive mobile robots to autonomously wipe tables to clean spills and crumbs. This problem is challenging, as it requires planning wiping actions while reasoning over uncertain latent dynamics of crumbs and spills captured via high-dimensional visual observations. Simultaneously, we must guarantee constraints satisfaction to enable safe deployment in unstructured cluttered environments. To tackle this problem, we first propose a stochastic differential equation to model crumbs and spill dynamics and absorption with a robot wiper. Using this model, we train a vision-based policy for planning wiping actions in simulation using reinforcement learning (RL). To enable zero-shot sim-to-real deployment, we dovetail the RL policy with a whole-body trajectory optimization framework to compute base and arm joint trajectories that execute the desired wiping motions while guaranteeing constraints satisfaction. We extensively validate our approach in simulation and on hardware. Video of experiments: https://youtu.be/inORKP4F3EI
BibTeX
@inproceedings{lew2023robotic,
  author = {Thomas Lew and Sumeet Singh and Mario Prats and Jeffrey Bingham and Jonathan Weisz and Benjie Holson and Xiaohan Zhang and Vikas Sindhwani and Yao Lu and Fei Xia and Peng Xu and Tingnan Zhang and Jie Tan and Montserrat Gonzalez},
  title = {{Robotic Table Wiping via Reinforcement Learning and Whole-body Trajectory Optimization}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2023},
  doi = {10.1109/icra48891.2023.10161283},
}
Figure