Abstract
Robotic imitation learning faces a significant challenge in handling multimodal expert demonstrations while maintaining real-time inference speeds. Current solutions like Diffusion Models offer high precision but suffer from high computational costs due to iterative denoising. We present Energy Policy, a framework that achieves the performance of diffusion-based methods in a single forward pass. By utilizing an Energy MLP with Adaptive Layer Normalization (adaLN) and an energy score objective, our method runs up to 70x faster than standard diffusion baselines, enabling high-frequency control in dynamic environments.
Introduction
In robotic imitation learning, a policy must map visual observations to precise robot actions. A primary hurdle is multimodality: when an expert demonstrates multiple valid paths to a goal, standard regression models (using L1/L2 loss) often average these paths, leading to failure.
Existing approaches typically fall into two categories:
- Autoregressive Models: These discretize actions into tokens, often sacrificing fine-grained precision.
- Diffusion Models: These handle multimodality well but require dozens of iterative steps, making them too slow for real-time reaction in dynamic settings.
Energy Policy bridges this gap by providing a natively multimodal, continuous, and highly efficient "third way" for robotic control.
Methods
The Energy Policy architecture replaces iterative refinement with a direct sampling approach.
Energy Score Objective
Instead of standard regression, the model is trained using a "strictly proper scoring rule" called the energy score. During training, the model generates two independent action samples and compares them to the ground truth. This loss function incentivizes the model to produce samples that accurately represent the target distribution while maintaining the natural diversity (multimodality) of the expert data.
Architecture: Energy MLP with adaLN
The system employs a Transformer decoder to process observations, followed by a dedicated Energy MLP. To enable the model to sample from complex distributions in a single pass, the authors utilize Adaptive Layer Normalization (adaLN). This mechanism modulates internal features with a noise vector, allowing the model to explore different action modes effectively.
Experimental Results
The framework was evaluated across extensive simulated benchmarks (Robomimic, Franka Kitchen, MimicGen, PushT, Meta-World) and real-world tasks (Cup, Rabbit, Catch).
Efficiency and Speed
Energy Policy demonstrates a dominant speed-to-performance ratio. It is 22x to 70x faster than Diffusion Policy and 2.3x to 7x faster than CARP. On the PushT task, it achieved state-of-the-art results with only 7.7M parameters, a fraction of the size of typical diffusion models (>250M).
Success Rates and Real-World Performance
On the MimicGen benchmark, Energy Policy achieved a 10% higher success rate than diffusion-based multitask models. Its speed is particularly critical in dynamic tasks like "Catch," where the robot must intercept a moving object. Energy Policy succeeded 13/20 times, significantly outperforming Diffusion Policy (8/20), which struggled with the latency of iterative planning.
Discussion and Limitations
While highly efficient, the Energy Policy is sensitive to the energy loss coefficient ($\alpha$), with performance degrading if set too high. Additionally, the model can overfit if the Energy MLP is made too wide (e.g., >1024 channels) relative to the available demonstration data. Future work will explore scaling this architecture to larger, more diverse foundation datasets for robotics to determine if these efficiency gains persist at the billion-parameter scale.
Summary updated