TL;DR
TacMMs (Tactile Mobile Manipulators) integrate soft biomimetic tactile sensors onto mobile robots to solve the "last-centimeter" problem in warehouse automation. By using touch instead of just vision to adjust their pose, a pair of these robots can cooperatively lift objects—especially those with non-uniform weight distributions—with significantly higher reliability than vision-only systems.
What Problem Does This Paper Address?
Warehouse automation currently relies heavily on vision-based systems (like AGVs and forklifts) to locate and lift goods. However, vision is a "distal" modality; once a robot gets close enough to an object to lift it, the object often occludes the camera's view. Furthermore, vision systems struggle with calibration inaccuracies and cannot provide the precise contact information—such as the exact normal angle or contact pressure—required for a stable lift.
This is particularly problematic for cooperative lifting, where two or more robots must balance an object. If the robots don't align perfectly or apply uneven force, the object is likely to slip or tip, especially if its center of mass is off-center.
Methods
The authors propose the TacMM, a hybrid system comprising two main open-source components:
- DOTS (Distributed Organisation and Transport System): An omnidirectional mobile robot equipped with a raisable platform.
- TacTip: A soft, biomimetic optical tactile sensor. It consists of a 3D-printed soft dome with internal pins. An internal camera tracks the movement of these pins as the dome deforms against a surface.
The Control Loop
The system uses Behavior Trees (BTs) to manage a multi-stage lifting process:
- Approach: Robots use vision to get near the target.
- Tactile Servoing: The robot makes contact, and a convolutional neural network (PoseNet) estimates the relative pose (angle and depth) from the tactile image.
- Adjustment: Based on tactile feedback, the robot backs off, rotates to align its sensor normal to the object surface, and re-contacts. This repeats until the alignment error is below a threshold.
- Cooperative Lift: Once both robots are aligned, they communicate to trigger a simultaneous vertical lift.
Datasets
To train the PoseNet model, the researchers used a Dobot Magician robot arm to collect a custom dataset:
- Tactile Images: 5,500 images of the TacTip pressing against a flat surface.
- Labels: Each image was labeled with a specific contact depth (1–5 mm) and angle ($\pm 25^\circ$).
- Robustness: The data included random horizontal and rotational shear to ensure the network learned to ignore "messy" contact dynamics and focus on the underlying pose.
- Non-contact Data: 500 images were taken at zero depth to help the system distinguish between "touching" and "not touching."
Key Insights and Ideas
- Touch Overcomes Occlusion: While vision gets the robot to the "ballpark," tactile sensing provides the high-fidelity local information needed for the final, critical alignment where vision fails due to proximity.
- Depth is the Secret Sauce: The authors found that the ability to accurately predict and control contact depth was the primary reason for the TacMM's success. It allows robots to apply equal and opposite forces, satisfying the Force Closure Grasp (FCG) condition necessary for a stable lift.
- Biomimetic Advantage: The TacTip’s soft, compliant nature (mimicking human skin) provides a larger contact area and inherent mechanical stability compared to rigid end-effectors.
Key Results
The TacMM system was compared against a vision-only baseline using ArUco markers for pose estimation:
- Overall Success Rate: TacMMs achieved a 66% success rate, more than double the vision baseline's 32%.
- Non-Uniform Loads: The performance gap widened significantly with unbalanced loads. For a 500g payload placed on top of an empty box, TacMM maintained a 50% success rate, while the vision system plummeted to 10%.
- Pose Accuracy: The PoseNet model was highly accurate, with a Mean Absolute Error (MAE) of just 0.26 mm in depth and 1.06° in angle.
Limitations and Future Work
- Minimal Configuration: This study only used two robots. While effective for simple boxes, two contact points are prone to letting the object rotate or slip.
- Scaling to Swarms: The authors intend to extend this to three or more robots, which would allow for more complex "caging" and manipulation of irregular objects.
- Dynamic Manipulation: Future work will explore using tactile feedback not just for the initial lift, but for active re-balancing and "in-hand" manipulation during transport.
- Slip Detection: The current system does not explicitly detect slip in real-time; integrating slip-detection modalities into the TacTip could further improve robustness.
Summary updated