Grounded Action Transformation for Sim-to-Real Transfer

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 217 papers24 conceptsfrozen 2026-10-04

When we train a robot in a simulator, we are training it in a dream. In this dream, joints are frictionless, sensors have zero latency, and surfaces are perfectly flat. When the robot "wakes up" in the real world, these small discrepancies—collectively known as the reality gap—cause the policy to fail.

Traditionally, researchers tried to close this gap through "System Identification": painstakingly tuning the simulator's physics parameters (like mass or friction) to match reality. However, many simulators are "black boxes" where internal parameters are inaccessible or too complex to tune. A more elegant solution has emerged: instead of fixing the simulator's physics, we can learn to transform the actions sent to it.

The Core Intuition: Changing the Input, Not the Engine

The fundamental idea behind Grounded Action Transformation (GAT), introduced by Josiah P. Hanna, Peter Stone, and their colleagues, is to treat the simulator as a target that needs to be "grounded" to real-world observations.

Imagine a robot wants to move its arm 10 degrees. In the real world, due to gravity and motor slack, an "action" of 10 degrees might only result in a 7-degree movement. If the simulator is "perfect," an action of 10 degrees results in exactly 10 degrees. GAT bridges this by finding a transformation: it learns that to get the "perfect" simulator to behave like the "messy" real world, it should pass an action of 7 degrees to the simulator instead of 10.

By training the policy inside this "grounded" simulator, the agent learns to compensate for real-world physics before it ever touches a physical motor. This approach allowed Josiah P. Hanna et al. to achieve the fastest stable walk for the NAO humanoid robot by simply grounding the simulator's action space.

The Mechanism: Forward and Inverse Models

To build this transformation, researchers typically use a two-step modeling process:

  1. The Forward Model: We collect a small amount of real-world data to learn a model that predicts the next state $s'$ given a current state $s$ and action $a$. This represents how the real world behaves.
  2. The Inverse Simulator Model: We learn a model of the simulator. Given a starting state $s$ and the desired next state $s'$ (predicted by our real-world forward model), what action $a_{sim}$ would the simulator need to take to get there?

As Igor Mordatch and Pieter Abbeel demonstrated in their early work on deep inverse dynamics, this allows us to use the simulator to decide where to go (the target state) while using a learned model to decide how to get there (the specific action).

Evolution: From Deterministic to Adversarial Grounding

The basic GAT framework has been extended to handle the complexities of the physical world:

  • Handling Noise: Real-world physics is often stochastic. Siddharth Desai and Garrett Warnell developed SGAT (Stochastic GAT), which teaches the simulator to "hallucinate" the same noise and uncertainty found in reality, leading to more robust policies.
  • Adversarial Grounding: Instead of simple regression, Ishan Durugkar and Josiah Hanna proposed GARAT. This uses an adversarial framework where a "transformer" tries to modify simulator actions so that a discriminator cannot tell the difference between a simulated trajectory and a real-world one.
  • High-Dimensional Policies: To scale these ideas to complex neural network controllers, Haresh Karnan et al. introduced RGAT (Reinforced GAT), which frames the grounding process itself as a reinforcement learning problem.

Beyond Robotics: Common Sense and Meta-Learning

The concept of action translation is now moving beyond pure robotics into broader domains. In traffic signal control, Longchao Da and Hua Wei developed PromptGAT. This framework uses Large Language Models (LLMs) to inject "common sense" physics—such as how snow or rain affects vehicle braking—into the action transformation loop, helping traffic policies adapt to weather conditions that the simulator never explicitly modeled.

Furthermore, Yijie Guo and Honglak Lee have shown that this isn't just for sim-to-real. Their MCAT framework uses action translation to help agents transfer skills between different simulated tasks. By learning how an action in a "source" task maps to an equivalent outcome in a "target" task, agents can reuse successful strategies even when the underlying physics change.

Summary

Grounded Action Transformation works because it decouples high-level intent from low-level execution. The policy focuses on the "intent" (what state should I reach?), while the transformation layer handles the "execution" (what specific command does this specific environment need to reach that state?). By shifting the burden of alignment from the simulator's source code to a learned action mapping, we can bridge the reality gap with far less data and manual engineering.

Papers in this version

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 11 papers
  2. Version 1 · 2026-10-04 · 17 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_action_transformation_3ba604c5,
  title = {action transformation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/action-transformation/rdr_3ba604c52b314255806f5e0ba18042a4},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_grounded_action_transformation_for_sim_t_3ba604c5_v1,
  title = {Grounded Action Transformation for Sim-to-Real Transfer},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/grounded-action-transformation-for-sim-to-real-transfer/rdr_3ba604c52b314255806f5e0ba18042a4/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}