Estimating Clean-up Robots' Mechanical Operations of Objects Using a SLAM Based Method

Si Gao, Naotomo Tatematsu, Jun Ohya, Zhao WangThe Abstracts of the International Conference on Advanced Mechatronics Toward Evolutionary Fusion of IT and Mechatronics ICAM 2015 · DOI

Abstract

To enable clean-up robots to vacuum cluttered floors, the robots should be able to return objects on the floors to the proper positions by pushing the objects. This paper proposes a computer vision based method that can detect the objects (clutter) and evaluate the result of the mechanical operation (pushing the objects). First, we construct a dense 3D map of a tidy situation of the room (after-clean-map). Next, we construct another dense 3D map of the cluttered situation of the same room (before-clean-map). Then, we construct the 3D point cloud of the cluttered situation by comparing the after-clean-map and the before-clean-map. Finally, we let a clean-up robot push the cluttered object and evaluate whether the object is successfully moved. Experimental results using real RGBD image sequences demonstrate promising results of our proposed method.

Introduction

While commercial cleaning robots like the Roomba are effective at vacuuming open floor spaces, they are often stymied by household clutter. Currently, these robots either collide with misplaced objects or navigate around them, leaving the floor beneath uncleaned. To achieve true autonomy, a robot must perform a "tidy-up" task: identifying misplaced items, returning them to their original locations, and then cleaning the vacated area.

This research addresses two primary challenges in this workflow: identifying what doesn't belong (clutter detection) and verifying if a mechanical operation, such as pushing an object, was successful without relying on expensive force sensors.

Proposed Method

The researchers proposed a four-step pipeline integrated with a mobile robot equipped with a Kinect RGB-D sensor.

Overview of the proposed four-step pipeline for clutter detection and evaluation.
  1. Dense 3D Mapping: The robot creates two dense 3D maps using SLAM: a "tidy" map ($M_A$) of the room in its ideal state and a "cluttered" map ($M_B$) where objects have been added.
  2. Change Detection: By aligning these maps using the Iterative Closest Point (ICP) algorithm, the system extracts a 3D point cloud of the "clutter." The method improves upon existing techniques by incorporating color information and specific noise filters to detect thin or flat objects.
  3. Mechanical Operation: The robot pushes the detected clutter toward its original coordinates.
  4. Evaluation: The robot estimates the actual distance moved ($d$) using three metrics: Wheel Odometry, Visual Odometry (tracking features in the camera frame), and Centroid Change (comparing 3D center-points before and after the push).

Experimental Results and Analysis

The study utilized original RGB-D image sequences to test the accuracy of clutter detection and the reliability of distance estimation methods across different object types and weights.

Graph showing the impact of object weight on the accuracy of different distance estimation methods.

Key Insights

  • Color-Enhanced Detection: Adding color data significantly improves the detection of thin clutter, such as a piece of paper on the floor, which geometric-only methods might ignore as sensor noise.
  • Shape-Dependent Sensing Strategy: The most effective way to track an object depends on its physics. For box-like objects, visual odometry is most accurate (average error of 3.3% to 3.6%). For ball-like objects, the centroid change method is superior because the robot's movement does not correlate with the ball's final position.
  • The Failure of Wheels: As objects get heavier (approaching 10kg), wheel odometry becomes highly unreliable due to tire slippage. This highlights why computer vision is essential for "feeling" the results of a mechanical operation.

Conclusion

The proposed SLAM-based method successfully detects clutter and evaluates the results of pushing operations. While the current system requires manual path control and a pre-existing "tidy" map, it provides a foundation for future autonomous tidy-up functions. Future work will focus on automating path planning and handling more complex, irregular household objects.

Summary updated

Abstract

To enable clean-up robots to vacuum cluttered floors, the robots should be able to return objects on the floors to the proper positions by pushing the objects. This paper proposes a computer vision based method that can detect the objects (clutter) and evaluate the result of the mechanical operation (pushing the objects). First, we construct a dense 3D map of a tidy situation of the room (after-clean-map). Next, we construct another dense 3D map of the cluttered situation of the same room (before-clean-map). Then, we construct the 3D point cloud of the cluttered situation by comparing the after-clean-map and the before-clean-map. Finally, we let a clean-up robot push the cluttered object and evaluate whether the object is successfully moved. Experimental results using real RGBD image sequences demonstrate promising results of our proposed method.
BibTeX
@inproceedings{gao2015estimating,
  author = {Si Gao and Naotomo Tatematsu and Jun Ohya and Zhao Wang},
  title = {{Estimating Clean-up Robots' Mechanical Operations of Objects Using a SLAM Based Method}},
  booktitle = {The Abstracts of the International Conference on Advanced Mechatronics Toward Evolutionary Fusion of IT and Mechatronics ICAM},
  year = {2015},
  doi = {10.1299/jsmeicam.2015.6.249},
}
Figure