TL;DR
This paper introduces a novel framework, Classifier-Free Guidance Diffusion Policy (CFG-DP), that enhances robotic manipulation in sequential tasks by explicitly modeling temporal progression. By integrating Classifier-Free Guidance (CFG) with timestep inputs, CFG-DP significantly improves task termination accuracy and reduces repetitive actions, leading to more reliable and deterministic robot control.
What Problem Does This Paper Address?
Robots performing sequential tasks, especially those with repetitive cycles like screwing, often struggle with precise termination. Existing methods like Diffusion Policy (DP) and Action Chunking with Transformers (ACT) primarily rely on short-term sensory inputs, lacking the temporal context to accurately judge when a task cycle is complete. This can lead to robots getting stuck in "local optima traps," performing excessive repetitive actions, and failing to terminate tasks reliably. This is a critical issue for practical applications where deterministic and dependable task execution is paramount.
Methods
The core innovation is the Classifier-Free Guidance Diffusion Policy (CFG-DP). This framework builds upon standard Diffusion Policies (DP) by incorporating two key elements:
- Classifier-Free Guidance (CFG): Inspired by previous work in goal-conditioned imitation learning, CFG is used to guide the action generation process. It involves training both a conditional diffusion model (which uses sensory inputs and timestep information) and an unconditional model (which only uses sensory inputs). During inference, these two models are combined using a dynamic guidance factor ($\lambda$) to steer the policy towards desired outcomes, particularly termination.
- Timestep Input ($S_t$): The model explicitly takes the current timestep count as an input. This allows the robot to track its progress through a task cycle. The timestep input is embedded and fused with other sensory information.
- Dynamic Guidance Factor ($\lambda$): The guidance factor $\lambda$ is dynamically adjusted based on the timestep count. It increases as the task approaches its expected completion time, giving more weight to the conditional model and thus emphasizing termination actions. This is implemented using a sigmoid function to ensure a smooth and adaptive increase in guidance.
The architecture uses a ResNet18 backbone for visual processing and fuses these features with joint states and the timestep count. The model predicts a sequence of actions over a defined horizon.
Datasets
The paper uses a dataset collected via VR-based teleoperation for a screwing task. This dataset consists of 200 demonstration trajectories, each annotated with timestep counts. The trajectories include variations in initial workpiece positions, multiple screwing cycles, and different termination positions to ensure robustness. The dataset was split into 80% for training and 20% for validation.
Key Insights and Ideas
- Temporal Context is Crucial for Termination: The paper strongly demonstrates that explicitly modeling temporal progression (via timestep counts) is essential for robots to accurately terminate repetitive tasks, a capability lacking in standard DP and ACT.
- CFG as a Termination Enhancer: CFG, when combined with temporal inputs, proves to be a powerful mechanism for guiding diffusion models towards precise task completion, effectively suppressing repetitive actions.
- Dynamic Guidance Adapts to Task Phase: The dynamic adjustment of the guidance factor ($\lambda$) based on the timestep allows the policy to be more exploratory early in a task and more deterministic near the termination point, mimicking human intuition.
- Humanoid Robots Benefit Greatly: The framework shows significant promise for improving the reliability and determinism of humanoid robots in complex, sequential manipulation tasks.
Key Results
- Real-World Screwing Task Performance: CFG-DP achieved a success rate of 83.2%, significantly outperforming DP (55.6%) and ACT (50.3%) on a real-world humanoid robot screwing task.
- Reduced Repetitive Actions: CFG-DP nearly eliminated repetitive actions, averaging only 0.3 redundant rotations per rollout, compared to 2.6 for DP and 3.2 for ACT.
- Faster Completion Time: CFG-DP completed tasks in an average of 24.2 seconds, compared to 32.0 seconds for DP and 41.0 seconds for ACT.
- Improved Termination Precision: Analysis of termination step distributions showed CFG-DP's distribution was more concentrated around the expected termination point, indicating superior precision.
- Ablation Studies: Qualitative analysis of conditional entropy showed that CFG-DP achieved earlier and more pronounced convergence towards deterministic actions near termination compared to configurations without CFG or timestep inputs.
Limitations and Future Work
- Timestep Count Estimation Sensitivity: The authors acknowledge that sensitivity to timestep count estimation remains a challenge.
- Future Work Directions:
- Developing more robust models for timestep count prediction, potentially through reinforcement learning.
- Optimizing timestep count prediction and reducing reliance on manual parameter tuning.
- Extending the framework to more complex, hierarchical, or modular tasks (e.g., assembly, pick-and-place) by decomposing them into sub-tasks.
- Investigating multi-task learning to adapt policies across diverse operations without retraining.
Summary updated