Introduction
Learning from Demonstration (LfD) has emerged as a powerful paradigm for teaching robots complex manipulation tasks. However, a significant bottleneck remains: generalization to objects with varying physical properties. A robot trained to wipe a surface with a soft sponge may fail when presented with a stiff or high-friction alternative, as the required force profiles and trajectories differ significantly. Traditional approaches to address this variability require exhaustive human demonstrations across the entire parameter space of object properties, which is often impractical. This paper presents a semi-supervised framework that enables robots to adapt to diverse physical properties using only a few human demonstrations by pre-training a haptic representation in simulation.
Methodology: Decoupling Sensing from Execution
The core innovation of this work is the decoupling of haptic perception from task execution. The framework consists of two primary components: a Haptic Representation Encoder and a Motion Generation Decoder.
Haptic Representation Encoder
The encoder is a Variational Auto-Encoder (VAE) designed to compress high-dimensional force-torque time-series data into a low-dimensional latent space. This model is trained using unsupervised data collected in the ROS-PyBullet simulator. The robot performs two types of exploratory actions—pressing and lateral motion—on virtual objects with varying stiffness and friction coefficients.
By training on 1,000 simulated trajectories, the encoder learns a latent representation that naturally organizes objects by their physical characteristics (e.g., grouping "stiff" vs. "compliant" objects) without explicit labels. The study revealed that "pressing" is optimized for identifying stiffness, while "lateral motion" is essential for surface friction.
Motion Generation Decoder and Few-Shot LfD
Once the haptic "sense" is pre-trained, a decoder is trained to map these latent representations to specific task trajectories. Unlike the encoder, the decoder is trained using a small set of human demonstrations (only 12 examples) on a physical KUKA iiwa robot. During the execution phase, the robot first performs a brief exploration to identify the object's latent representation, which then conditions the decoder to generate the appropriate wiping motion.
Experimental Results
The proposed method was evaluated on a surface-wiping task using sponges of varying stiffness and friction. Key findings include:
- Accuracy: The pre-trained model achieved a 50% reduction in Root Mean Squared Error (RMSE) for generated motions compared to a baseline LfD model without haptic pre-training.
- Extrapolation: The framework demonstrated robust performance on objects with properties outside the range of the human demonstrations. For instance, in stiffness extrapolation, the proposed model maintained an RMSE of $6.4 \pm 3.6$, whereas the baseline error spiked to $25.1 \pm 22.9$.
- Sim-to-Real Transfer: Despite being trained on simulated haptic data, the encoder successfully generalized to real-world objects, indicating that the fundamental physics of force interaction are well-captured in the latent space.
Conclusion
By leveraging unsupervised simulation data to build a foundational haptic representation, this research significantly reduces the human effort required for robot training. The ability to generalize from a handful of demonstrations to a wide array of physical object properties marks a significant step toward more adaptable and efficient robotic assistants. Future work will explore active perception strategies and online adaptation to dynamic object properties.
Summary updated