Learning More With Less: Sample Efficient Dynamics Learning and Model-Based RL for Loco-Manipulation

Benjamin Hoffman, Jin Cheng, Chenhao Li, Stelian CorosarXiv 2025 · DOI · arXiv

Abstract

Loco-manipulation—the coordination of legged locomotion and robotic arm manipulation—presents a significant challenge due to complex, non-smooth dynamics and the "black-box" nature of commercial hardware like the Boston Dynamics Spot. This paper proposes a sample-efficient framework that leverages a hand-crafted kinematic model as a physical prior within a Bayesian Neural Network (BNN). Using Functional Stein Variational Gradient Descent (FSVGD), the system learns to bridge the gap between simplified physics and real-world observations. Our results demonstrate that this "informed" dynamics model enables high-precision trajectory tracking with significantly less data than traditional model-free or purely data-driven methods, achieving over 140% improvement in tracking accuracy in low-data regimes.

Introduction

Controlling a legged robot equipped with a manipulator is inherently difficult. The movement of the base introduces disturbances to the arm, and the arm's movement shifts the robot's center of mass. While platforms like the Boston Dynamics Spot are highly capable, their internal controllers are proprietary, making exact analytical modeling impossible. Researchers typically choose between oversimplified hand-crafted models or data-hungry model-free Reinforcement Learning (RL).

Boston Dynamics Spot performing loco-manipulation tasks.

This work addresses the data efficiency gap by learning a "world model" that incorporates physical priors. By treating physics as a "compass" rather than a perfect map, the proposed framework enables the robot to learn complex behaviors from just a few minutes of real-world interaction.

Methodology

The framework employs a two-stage model-based RL approach:

  1. Informed Dynamics Learning (SIM-FSVGD): The core innovation is the use of a "sim-to-real" prior. A low-fidelity kinematic model based on Forward Euler integration provides a baseline for the robot's expected movement. A BNN, trained via FSVGD, then learns the residual dynamics—the complex wrenches and non-linearities not captured by the simple model.
  2. Policy Optimization: With an accurate dynamics model in hand, a control policy is trained using Soft Actor-Critic (SAC). This allows for efficient policy derivation in simulation, minimizing the need for hardware iterations.

To ensure real-world stability, the model accounts for a 133ms control latency by including action history in the state vector. Data was collected at 15 Hz using manual control (Xbox controller for the base, 3D Space Mouse for the end-effector).

Experimental Results

The system was evaluated on a Boston Dynamics Spot using datasets ranging from 500 to 5,000 points.

Model Accuracy and Sample Efficiency

The proposed SIM-FSVGD method consistently outperformed both standard FSVGD (no prior) and a purely fitted kinematic model (SIM-MODEL). As shown in the validation metrics, the informed model achieves significantly lower Negative Log-Likelihood (NLL), particularly when data is scarce ($N < 3000$).

Quantitative performance metrics for model validation and trajectory tracking.

Trajectory Tracking Performance

In real-world hardware tests, the robot was tasked with tracking 3D ellipsoidal and helical paths. At $N=2000$ datapoints, the baseline methods (FSVGD and SIM-MODEL) exhibited tracking errors that were 147% and 153% higher, respectively, than the SIM-FSVGD approach. This highlights the massive "head start" provided by the physical prior, allowing the robot to achieve high precision with roughly 80% less data than required by standard learning methods.

Conclusion

This research demonstrates that integrating even low-fidelity physical priors into Bayesian learning frameworks can drastically reduce the data requirements for complex robotic tasks. While current work focuses on 3D position tracking, future iterations will incorporate end-effector orientation and explore the robot's dynamic limits. By bridging the gap between classical robotics and modern machine learning, we move closer to robots that can learn to interact with the world as efficiently as biological systems.

Summary updated

Abstract

By combining the agility of legged locomotion with the capabilities of manipulation, loco-manipulation platforms have the potential to perform complex tasks in real-world applications. To this end, state-of-the-art quadrupeds with manipulators, such as the Boston Dynamics Spot, have emerged to provide a capable and robust platform. However, the complexity of loco-manipulation control, as well as the black-box nature of commercial platforms, pose challenges for deriving accurate dynamics models and robust control policies. To address these challenges, we turn to model-based reinforcement learning (RL). We develop a hand-crafted kinematic model of a quadruped-with-arm platform which - employing recent advances in Bayesian Neural Network (BNN)-based learning - we use as a physical prior to efficiently learn an accurate dynamics model from limited data. We then leverage our learned model to derive control policies for loco-manipulation via RL. We demonstrate the effectiveness of our approach on state-of-the-art hardware using the Boston Dynamics Spot, accurately performing dynamic end-effector trajectory tracking even in low data regimes. Project website and videos: https://sites.google.com/view/learning-more-with-less.
BibTeX
@misc{hoffman2025learning,
  author = {Benjamin Hoffman and Jin Cheng and Chenhao Li and Stelian Coros},
  title = {{Learning More With Less: Sample Efficient Dynamics Learning and Model-Based RL for Loco-Manipulation}},
  year = {2025},
  eprint = {2501.10499},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.10499},
}
Figure