TL;DR
DARE (Disturbance-Aware REtrieval) is a new framework that allows robots to retrieve objects from dense clutter by intentionally—and safely—interacting with surrounding items. By combining a geometry-aware graph neural network with a novel "retrospective correction" mechanism, DARE can predict complex contact cascades and fix long-horizon simulation drift using sparse, noisy observations.
What Problem Does This Paper Address?
Imagine trying to grab a specific bottle of juice from the back of a packed refrigerator. You can’t lift it over the other items because of the shelf above, so you have to slide it out, inevitably bumping into other jars and cartons. If you aren't careful, a small nudge to a mustard jar might tip over a delicate wine glass nearby.
For robots, this is a nightmare. Traditional "collision avoidance" planners simply give up if there's no clear path. "Direct retrieval" planners might reach the goal but leave a trail of toppled objects in their wake. While learned dynamics models can predict these interactions, they suffer from accumulated drift: small errors in predicting the first contact compound over time, making long-term plans unreliable. Furthermore, real-world sensor data is often sparse and noisy, making it difficult to simply "reset" a simulation to the current observed state without introducing physical inconsistencies.
Methods
The authors introduce DARE, which tackles these issues through three main components:
- Geometry-Aware Relational Dynamics: Instead of using slow, high-resolution meshes or overly simple bounding boxes, DARE decomposes objects into convex pieces and encodes them into compact descriptors. These descriptors are fed into a Propagation Network (a type of Graph Neural Network) that models how forces ripple through a chain of objects.
- Retrospective Correction: This is the core innovation. When the robot receives a new (potentially noisy) observation, it doesn't just overwrite its current state. Instead, a learned "corrector" looks back at the discrepancy between the prediction and the observation. It goes "back to the future" by amending an earlier predicted state and rerunning the dynamics forward. This ensures the resulting trajectory is physically consistent with the model's learned rules.
- Joint Predictor-Corrector Learning: The dynamics model and the corrector are trained together. This encourages the dynamics model to produce rollouts that are "correctable" and the corrector to suggest amendments that actually improve future predictions, not just fit the current data point.
These predictions feed into a Model Predictive Path Integral (MPPI) planner that balances the progress toward the target object against the predicted "disturbance" (displacement, tilt, and rotation) of bystanders.
Datasets
The researchers evaluated DARE across three increasingly complex simulated environments:
- Planar Puck Table: A 2D environment with five pucks used for basic dynamics validation.
- RoboCasa Tabletop: 3D scenes with diverse household objects.
- RoboCasa Kitchen Cabinets: The most challenging setup, featuring 13 different compartment geometries and varying levels of clutter (up to 20 objects).
The evaluation specifically included "unseen" objects—items the model never encountered during training—to test its ability to generalize based on geometry alone.
Key Insights and Ideas
- The "Correctability" Objective: Training the model to be corrected is a powerful paradigm. By supervising the "rerolled" trajectory against a ground-truth future, the model learns to prioritize amendments that fix the root cause of an error rather than just smoothing over the symptoms.
- Physics-Consistent Filtering: Simply snapping a simulation to a noisy observation often breaks the physics (e.g., objects might suddenly overlap). By amending a past state and rerunning the learned dynamics, DARE keeps the objects in a physically plausible configuration.
- Geometry over Resolution: The paper demonstrates that compact descriptors derived from convex hulls are sufficient for predicting complex contact dynamics, avoiding the massive computational overhead of mesh-based Graph Neural Networks.
Key Results
DARE significantly outperformed existing baselines in both prediction accuracy and retrieval success:
- Lower Error: In long-horizon (40-step) predictions, DARE achieved the lowest normalized terminal translation and rotation errors across all RoboCasa test sets.
- Higher Success in Clutter: In "Hard" cabinet scenarios with 20 objects, DARE maintained a success rate of 0.90 (with clean observations), whereas Collision Avoidance dropped to 0.10 and Direct Retrieval to 0.20.
- Noise Robustness: Even with significant observation noise (2cm position error and 13° rotation error), DARE's retrospective correction allowed it to maintain high success rates where "Direct Observation" updates failed due to noise injection.
Limitations and Future Work
While DARE is a major step forward, the authors note some areas for growth:
- Geometry Knowledge: The current system assumes the robot has access to the collision geometry of the objects (though it generalizes to new shapes). Integrating this with real-time vision-based geometry estimation would be a logical next step.
- Tactile Feedback: While the model uses visual observations for correction, incorporating high-frequency tactile or force feedback during the "contact-rich" phases could further improve the precision of the retrieval.
- Computational Scaling: While more efficient than mesh-based methods, the GNN's message-passing cost still scales with the number of objects, which might limit its use in extremely high-density environments (e.g., a bin of hundreds of small parts).
Summary updated