Closed-Loop Visuomotor Control with Generative Expectation for Robotic Manipulation

Qingwen Bu, Jia Zeng, Li Chen, Yanchao Yang, Guyue Zhou, Junchi Yan, Ping Luo, Heming Cui, Yi Ma, Hongyang Li

Introduction

Most modern robotic agents operate on an "open-loop" paradigm: they receive an instruction, generate a sequence of actions, and execute them without verifying if the environment has changed or if they have drifted off course. This leads to the "compounding error" problem, where a minor mistake at the start of a long task—such as grabbing an item from a shelf—can lead to total failure by the end.

CLOVER (Closed-Loop Visuomotor Control with Generative Expectation) is a new framework that brings the principles of classic control theory to modern robotic manipulation. By using a video diffusion model to generate "visual expectations" and a novel measurable embedding space to quantify errors, CLOVER allows robots to adaptively transition between sub-goals and trigger replanning when they encounter unexpected obstacles or execution errors.

Methodology

The CLOVER framework consists of three main pillars that transition robotic control from open-loop execution to a feedback-driven system.

The CLOVER framework (b) compared to classic closed-loop control (a).

1. Visual Planner (Generative Expectation)

The system uses a text-conditioned video diffusion model to generate a sequence of future RGB-D frames. These frames serve as the "reference input" or the visual expectation of what the task should look like. To ensure these plans are physically realistic, the authors incorporate optical flow regularization and depth information, which helps the model understand 3D geometry and motion consistency.

2. Measurable Embedding Space

A core innovation of CLOVER is the learned state embedding space. Instead of using generic visual features like CLIP, which are effective for semantic identification but poor at measuring physical distance, CLOVER learns a specific representation where the cosine distance between the current observation and the goal observation directly reflects task progress.

3. Feedback-Driven Controller

An MLP-based policy decodes the "error" (the difference between current and goal embeddings) into robot actions. The system follows an Adaptive Transition & Replan logic: it only moves to the next sub-goal once the current one is reached (i.e., the error is low), and it triggers a total replan if the generated sub-goals are detected as unreachable or inconsistent.

Detailed architecture of the CLOVER state encoder, error measurement, and action decoder.

Experimental Results

The authors evaluated CLOVER on the CALVIN benchmark—a simulated environment for long-horizon, language-conditioned manipulation—and in real-world robotics experiments using an AIRBOT Play robotic arm.

Performance and Robustness

  • Benchmark Success: CLOVER set a new state-of-the-art on the CALVIN benchmark, improving the success rate for completing 5-task chains by 8% over previous open-loop methods.
  • Real-World Reliability: In real-world tests, CLOVER nearly doubled the average task completion length compared to Google’s RT-1.
  • Environmental Resilience: The system proved remarkably resilient to "visual distractions" (random new objects) and "dynamic scenes" (moving objects during execution), where baseline models like RT-1 and ACT saw their success rates plummet.

Conclusion

CLOVER demonstrates that integrating generative visual expectations with measurable error feedback significantly enhances the robustness of robotic manipulation. By "closing the loop," robots can better handle the uncertainties of real-world environments. Future work will explore pre-training on larger "actionless" video datasets and optimizing inference speeds for higher-frequency control.

Summary updated

BibTeX
@inproceedings{bu2024closed,
  author = {Qingwen Bu and Jia Zeng and Li Chen and Yanchao Yang and Guyue Zhou and Junchi Yan and Ping Luo and Heming Cui and Yi Ma and Hongyang Li},
  title = {{Closed-Loop Visuomotor Control with Generative Expectation for Robotic Manipulation}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=1ptdkwZbMG},
}
Figure