TL;DR
This paper introduces Being-H0.5, a foundational Vision-Language-Action (VLA) model that achieves robust cross-embodiment generalization for robots. By leveraging a massive new dataset (UniHand-2.0) and a novel "human-centric" learning approach, Being-H0.5 enables a single robot policy to operate effectively across diverse robotic platforms, even those it hasn't seen before.
What Problem Does This Paper Address?
Scaling robotic intelligence to perform a wide range of tasks across different robot bodies (embodiments) is a major challenge. Current VLA models often struggle with this "cross-embodiment generalization" due to:
- Morphological Heterogeneity: Robots have vastly different physical forms, joint structures, and control mechanisms, making it hard for a single model to adapt.
- Data Scarcity: Collecting enough diverse robot data for every possible platform and task is prohibitively expensive and time-consuming.
- "Physical Gap": Models trained on simpler robots often fail when transferred to more complex ones because the underlying physics and action spaces are so different. This leads to unstable or incorrect movements.
The authors propose treating human interaction as a universal "mother tongue" for robots, believing that human movements capture fundamental physical principles and interaction logic that can be transferred to various robotic embodiments.
Methods
Being-H0.5 employs several key innovations:
- Human-Centric Learning: The core idea is to use human demonstrations as a primary source of learning. Human motion is seen as a "universal template" that captures generalizable physical priors.
- UniHand-2.0 Dataset: A massive new dataset (35,000+ hours, 120B tokens) combining human demonstrations, robot manipulation data from 30+ embodiments, and visual-language understanding data.
- Unified Action Space: A novel representation that maps diverse robot actions and human hand motions into a common, semantically aligned "physical vocabulary." This allows the model to learn general manipulation skills independent of specific robot hardware.
- Mixture-of-Transformers (MoT) Architecture: This architecture disentangles high-level reasoning (understanding) from low-level action generation, with a shared backbone for efficient cross-modal learning.
- Mixture-of-Flow (MoF): A scalable framework within the action generation expert that uses shared "foundation experts" and specialized "embodiment-specific experts" to handle diverse robot dynamics efficiently.
- Manifold-Preserving Gating (MPG): A technique to make policies robust to sensory shifts and noisy observations by dynamically adjusting reliance on context features versus a stable prior.
- Universal Async Chunking (UAC): A protocol to handle real-time inference and execution delays across robots with different latencies, ensuring smooth control.
Datasets
- UniHand-2.0: The paper introduces this large-scale dataset, comprising:
- 16,000 hours of egocentric human video.
- 14,000 hours of robot manipulation data from 30 distinct robotic embodiments.
- 5,000 equivalent hours of visual-language understanding data.
- Existing Benchmarks: The model is evaluated on:
- Simulated: LIBERO and RoboCasa.
- Real-World: Five distinct robotic platforms (upper-body humanoids, single-arm dexterous manipulators, and lightweight parallel grippers).
Key Insights and Ideas
- Human motion as a universal language: The paper provides strong evidence that human demonstrations are not just "extra data" but a fundamental source of transferable physical priors that enable generalization across diverse robots.
- Unified action space is crucial: By abstracting robot actions into a common space, the model can learn general manipulation skills that are not tied to specific hardware, leading to better cross-embodiment transfer.
- Scale and diversity matter: The massive scale and diversity of UniHand-2.0 are key to achieving robust generalization, allowing the model to learn robust representations of interaction dynamics.
- Zero-shot transfer is possible: The research demonstrates a notable embodiment-level zero-shot transfer phenomenon, where a single generalist checkpoint can perform tasks on unseen robot embodiments, hinting at a promising direction for future VLA research.
Key Results
- State-of-the-art on benchmarks: Being-H0.5 achieves state-of-the-art results on simulated benchmarks like LIBERO (98.9% average success) and RoboCasa (53.9% average success).
- Strong real-world performance: The generalist model (a single checkpoint trained on all embodiments) performs competitively with specialist models on real robots, showing only marginal degradation.
- Significant outperformance over baselines: Being-H0.5 significantly outperforms prior VLA models like π0.5, especially on challenging long-horizon and bimanual tasks.
- Embodiment-level zero-shot transfer: The generalist model shows a non-zero success rate on unseen task-embodiment pairs on real robots, demonstrating emergent generalization capabilities.
Limitations and Future Work
- Zero-shot success rate is lower: While zero-shot transfer is demonstrated, the success rate and motion precision are less reliable compared to in-distribution performance.
- Future work directions:
- Further scaling of post-training data diversity and coverage to improve emergent intelligence.
- Systematic benchmarking of alternative VLM backbones.
- Exploring more sophisticated embodiment-specific adaptation strategies.
- Investigating the impact of different architectural biases on robotic performance.
Summary updated