Introduction: The Curse of Too Much Data
In robotics, the prevailing wisdom often suggests that more sensors lead to more robust policies. However, this paper identifies a critical failure mode called state over-specification. When a policy is trained on multimodal data (e.g., RGB, depth, and proprioception), it may inadvertently rely on environment-specific noise—such as specific lighting or background textures in a simulation—that does not generalize to the real world.
While humans can easily distinguish between task-relevant information (the shape of an object) and irrelevant details (the color of a table), standard imitation learning (IL) models often overfit to these distractions. This leads to a significant performance drop when the robot is deployed in a new environment.
Masked Imitation Learning (MIL)
To address this, the authors propose Masked Imitation Learning (MIL), a framework designed to automatically discover which modalities are truly invariant across environments. MIL employs a bi-level optimization strategy to learn a binary mask ($\Psi$) that selectively filters input features before they reach the policy network.
Bi-Level Optimization
The MIL process consists of two nested loops:
- Inner-Level Optimization: For a given mask, the model learns a policy ($F_\theta$) by minimizing a standard behavioral cloning loss ($\mathcal{L}_{in}$) on the training dataset.
- Outer-Level Optimization: The binary mask is updated by evaluating the learned policy on a separate validation dataset. Instead of using simple per-step action errors, MIL utilizes a forward dynamics model ($\bar{T}$) to simulate full trajectories. The validation loss ($\mathcal{L}_{out}$) is calculated based on the distance between these "imagined" rollouts and the ground truth trajectories. This approach better accounts for compounding errors in long-horizon tasks.
The algorithm uses coordinate descent to iteratively flip bits in the binary mask, retaining configurations that improve validation performance. The authors found that hard binary masks significantly outperform soft, continuous masks, which often allow harmful noise to leak through.
Experimental Results and Insights
The authors evaluated MIL across several domains, including simulated manipulation tasks (Robomimic-Can and Square) and a real-world bookshelf reaching task.
- Significant Performance Boosts: In the complex Robomimic-Square task, MIL achieved a 71.3% success rate, a 5.6x improvement over vanilla Behavioral Cloning (12.7%).
- Sim-to-Real Robustness: On a physical Franka Panda robot, MIL reached a 95.24% success rate by correctly identifying and masking out unreliable camera views, whereas standard BC lagged at 54.17%.
- Discovery of Invariance: MIL successfully "discovered" oracle-like masks. For instance, it learned to ignore RGB data when depth and proprioception were sufficient and more stable across environments.
Conclusion and Future Directions
Masked Imitation Learning provides a principled way to handle multimodal data in robotics by filtering out distracting information. While the current bi-level optimization approach is computationally intensive and uses a static mask, it represents a major step toward building robots that can truly generalize. Future research may explore dynamic masking—where the robot adjusts its attention to different sensors at different stages of a task—and more efficient optimization techniques.
Summary updated