Introduction
Autonomous table wiping presents a significant "visuo-motor" challenge for robotics. Unlike humans, robots struggle with the unpredictable dynamics of crumbs and spills, which may stick to the wiper or be pushed around. Furthermore, a mobile manipulator must coordinate its base and arm movements—known as whole-body control—to reach across surfaces while avoiding obstacles like chairs and maintaining consistent contact force.
This paper introduces a hybrid framework that addresses these challenges by combining Reinforcement Learning (RL) for high-level strategic planning with Whole-body Trajectory Optimization for safe, constrained execution. By modeling messy dynamics using Stochastic Differential Equations (SDEs), the researchers trained a policy entirely in simulation that transfers "zero-shot" to real-world hardware.
Methodology
The proposed system decouples the cleaning task into a four-step pipeline, balancing intelligent planning with safety guarantees.
- Perception: LiDAR and camera data are processed to create a 64x64 binary image mask of "dirt" and bounding boxes for obstacles. This abstraction bypasses the visual sim-to-real gap.
- High-Level Planning (RL): A Soft Actor-Critic (SAC) policy analyzes the mask to select the optimal "wipe" primitive, defined by a start position, orientation, and length.
- Stochastic Modeling: To train the RL policy without real-world data, the authors developed a simulator using Stochastic Differential Equations (SDEs). This models particle movement (Brownian motion) and the probability of dirt being cleaned (Poisson process).
- Whole-body Trajectory Optimization: A "ShootingSQP" optimizer calculates joint trajectories for the robot's base and arm. It ensures collision avoidance and maintains a steady 10N force on the table via admittance control.
The RL policy was trained on 50 million environment steps in simulation and validated through 1,000 simulated rollouts before real-world testing on the Everyday Robots mobile manipulator platform.
Results and Key Insights
The hybrid approach demonstrated several advantages over traditional heuristic methods. The RL-based policy cleaned surfaces significantly faster by intelligently targeting areas of high dirt density.
Key findings include:
- Zero-Shot Transfer: The system successfully transitioned from simulation to real hardware without any fine-tuning.
- Robustness to Clutter: The robot successfully gathered crumbs and wiped spills while navigating around chairs that restricted its movement.
- Generalization: Despite being trained on simple dirt distributions, the policy effectively handled complex, multi-area spills in real-world environments.
By decoupling strategy from safety, the framework leverages the strengths of RL for "messy" logic while relying on trajectory optimization for provable safety and physical constraints.
Limitations and Future Work
While effective, the current system has limitations. The perception module relies on basic color filtering, which may fail in complex lighting or with low-contrast spills. Additionally, the robot is currently limited to straight-line wiping primitives.
Future research could explore:
- Curved Wiping: Developing more complex, non-linear wiping motions.
- Material Awareness: Learning SDE parameters (e.g., stickiness or absorbency) online by observing how spills react to initial contact.
- Kinematic Generalization: Adapting the trajectory optimization for different robotic platforms beyond the specific kinematics used in this study.
Summary updated