Disentangling Dynamics and Content for Control and Planning

Ershad Banijamali, Ahmad Khajenezhad, Ali Ghodsi, Mohammad GhavamzadeharXiv 2017 · arXiv

Introduction

In the fields of robotics and autonomous systems, interpreting the world through high-dimensional sensors like cameras is a fundamental challenge. To perform effective control, these complex visual inputs must be mapped into a simplified, low-dimensional "state space" where the effects of actions can be accurately predicted. While existing frameworks like "Embed to Control" (E2C) and "Robust Controllable Embedding" (RCE) have made strides in this area, they typically rely on datasets where every observation is paired with an explicit action label.

In practical scenarios, however, labeled data (set $X$) is often scarce, while raw, unlabeled observation data (set $Y$) is abundant. This paper introduces a novel method to bridge this gap by disentangling "dynamics" from "content," allowing a model to transfer physical laws learned from a small labeled dataset to a different, unlabeled environment that shares the same underlying physics.

Methodology: Disentangling Dynamics and Content

The core of the proposed approach is built upon Robust Controllable Embedding (RCE) and Variational Autoencoders (VAEs). The researchers propose a dual-latent architecture that separates the representation into two independent components:

  1. Dynamics Latent ($z$): Encodes the features necessary to describe movement and state transitions.
  2. Content Latent ($w$): Captures static visual features or "style" that do not influence the system's physics.
Architecture of the proposed model, showing the disentanglement of dynamics (z) and content (w) through encoder-decoder networks and a transition network.

To facilitate efficient control, the model enforces locally-linear dynamics within the latent space. The transition from one state to the next is governed by the linear equation $\hat{z}_{t+1} = A_t z_t + B_t u_t + c_t$. This linearity allows for the application of classical control algorithms, such as iterative Linear Quadratic Regulators (iLQR), directly in the latent space.

The critical innovation lies in a distributional constraint: the model forces the dynamics latent of the unlabeled set ($Y$) to match the distribution of the dynamics latent of the labeled set ($X$). By minimizing the KL divergence between these distributions, the system learns that despite visual differences, the underlying "engine" driving both datasets is identical.

Experimental Setup

The model was evaluated using a Planar System Domain, consisting of a 2D environment with six obstacles. The observations were $40 \times 40$ pixel images of an agent navigating the space. To test the transferability of the learned dynamics, the researchers used two distinct sets:

  • Set $X$ (Labeled): 8,000 samples including action triples $(x_t, u_t, x_{t+1})$, featuring an agent of a specific shape.
  • Set $Y$ (Unlabeled): 2,000 raw observations featuring an agent with a different shape and no associated action labels.

Results and Planning Performance

The experimental results demonstrate that the model successfully learned to map both datasets into a shared, physically consistent latent space. As shown in the visualization of the learned manifolds, the space for set $Y$ closely mirrors both the true state-space and the space estimated for set $X$.

Visualization of the true state-space versus the learned latent spaces for labeled set X and unlabeled set Y, alongside planning sequences.

Quantitatively, the model achieved parity in performance across both datasets:

Metric With Action ($X$) Without Action ($Y$)
Reconstruction Loss $3.6 \pm 1.7$ $3.9 \pm 2.2$
Prediction Loss $6.2 \pm 2.8$ $6.3 \pm 3.0$
Success Rate 100% 100%

Remarkably, the model achieved a 100% success rate in navigating the agent to a goal in the unlabeled environment ($Y$), despite never seeing an action label for those images during training. This "zero-shot" planning capability confirms that the model effectively transferred the learned physics from $X$ to $Y$.

Conclusion and Future Directions

This work highlights that "physics is universal, while style is local." By explicitly disentangling content from dynamics, the model can focus on shared physical properties for planning without being distracted by visual variations.

While the current model assumes identical underlying dynamics, future work could address "dynamics mismatch"—a common hurdle in Sim-to-Real transfer where friction or mass might differ. Additionally, scaling these methods to complex 3D environments remains a key objective. Potential applications are vast, ranging from self-driving cars that transfer knowledge between different sensor suites to robotic systems that learn from diverse visual demonstrations.

Summary updated

Abstract

In this paper, We study the problem of learning a controllable representation for high-dimensional observations of dynamical systems. Specifically, we consider a situation where there are multiple sets of observations of dynamical systems with identical underlying dynamics. Only one of these sets has information about the effect of actions on the observation and the rest are just some random observations of the system. Our goal is to utilize the information in that one set and find a representation for the other sets that can be used for planning and ling-term prediction.
BibTeX
@misc{banijamali2017disentangling,
  author = {Ershad Banijamali and Ahmad Khajenezhad and Ali Ghodsi and Mohammad Ghavamzadeh},
  title = {{Disentangling Dynamics and Content for Control and Planning}},
  year = {2017},
  eprint = {1711.09165},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1711.09165},
}
Figure