Abstract
Robotic manipulation of novel objects remains a significant challenge due to the complex, non-linear physics involved in interactions like pushing. We present CAZSL (Context-Aware Zero-Shot Learning), a framework that enables robots to generalize physical intuition to entirely new objects without requiring "practice" pushes. By utilizing a Siamese network architecture and a novel context-masking technique, CAZSL adapts its internal parameters based on an object's visual or structural features. Our results on the Omnipush dataset demonstrate that CAZSL achieves zero-shot generalization that outperforms standard meta-learning approaches, reducing prediction error by approximately 10%.
Introduction
Predicting the dynamics of robotic pushing is notoriously difficult. Factors like friction, center of mass, and geometry create complex, non-linear interactions. While deep learning models can be trained for specific objects, they often fail when encountering "out-of-distribution" (OOD) objects—those with different shapes or mass distributions than what was seen during training.
Current solutions typically rely on meta-learning, which requires a few "support samples" (trial pushes) to adapt to a new object. However, for many real-world applications, zero-shot performance—where the robot immediately understands an object's behavior upon sight—is preferred. CAZSL (pronounced "casual") addresses this by learning a continuous relationship between an object's "context" and its physical behavior.
Methods
The CAZSL framework is built upon three primary architectural pillars designed to facilitate generalization:
1. Context Masking
Rather than simply concatenating context variables (like shape or weight) with the input, CAZSL uses a separate "context network" to generate a mask. This mask is element-wise multiplied with the activations of the main prediction network. This effectively reconfigures the neural network to be specialized for the specific object being manipulated, allowing the context to fundamentally alter the model's "logic."
2. Siamese Network Structure and Regularization
During training, the model employs a Siamese structure, processing pairs of different objects simultaneously. This allows the model to learn relative differences in behavior based on differences in context. A critical component is Context Regularization, a loss term that ensures objects with similar contexts (e.g., similar geometry) produce similar masks and, consequently, similar predicted behaviors. The authors tested both standard Euclidean ($L_2$) distance and a "Neural Regularization" approach using a small MLP to learn the distance metric.
Experimental Evaluation
The researchers evaluated CAZSL using the Omnipush Dataset, a large-scale real-world dataset featuring 250 objects with diverse shapes and mass distributions collected via an industrial robot arm. Two types of context were tested:
- Indicator Context: A 36-bit binary vector describing the object's properties.
- Visual Context: $32 \times 32$ top-down images of the objects.
Key Insights
- Masking vs. Concatenation: Masking was found to be significantly more effective than simple concatenation for physical generalization.
- The Power of Vision: Using raw images as context consistently outperformed manual feature vectors, as convolutional layers could extract nuanced geometric features that humans might miss.
- Zero-Shot Efficiency: CAZSL often matched or exceeded the performance of meta-learning baselines like Attentive Neural Processes (ANP), despite the latter requiring extra samples from the new object to adapt.
Results
On the Omnipush dataset, CAZSL reduced the Root Mean Square Error (RMSE) by roughly 9-10% compared to standard fully connected networks. It demonstrated robust generalization to unseen mass distributions and maintained stable prediction errors across various surfaces and shapes (ranging from 4.2mm to 4.8mm), proving its robustness in diverse physical environments.
Conclusion and Future Work
CAZSL represents a significant step toward zero-shot physical reasoning in robotics. However, the current model is restricted to one-step horizon predictions; future work will focus on multi-step trajectory prediction where error compounding is a factor. Additionally, the authors aim to integrate CAZSL into closed-loop controllers to validate its effectiveness in real-time manipulation tasks involving more complex dynamics like tumbling or soft-body manipulation.
Citation: Zhang, W., Seto, S., & Jha, D. K. (2020). CAZSL: Zero-Shot Regression for Pushing Models by Generalizing Through Context. arXiv:2003.11696v2 [cs.LG].
Summary updated