Abstract
Granular media (e.g., cereal grains, plastic resin pellets, and pills) are ubiquitous in robotics-integrated industries, such as agriculture, manufacturing, and pharmaceutical development. This prevalence mandates the accurate and efficient simulation of these materials. This work presents a software and hardware framework that automatically calibrates a fast physics simulator to accurately simulate granular materials by inferring material properties from real-world depth images of granular formations (i.e., piles and rings). Specifically, coefficients of sliding friction, rolling friction, and restitution of grains are estimated from summary statistics of grain formations using likelihood-free Bayesian inference. The calibrated simulator accurately predicts unseen granular formations in both simulation and experiment; furthermore, simulator predictions are shown to generalize to more complex tasks, including using a robot to pour grains into a bowl, as well as to create a desired pattern of piles and rings.
Introduction
Granular materials are a persistent challenge for robotics. They do not behave strictly like solids or liquids, and their macroscale behavior—such as how a pile forms—is dictated by microscale properties like the friction between two tiny seeds. These properties are nearly impossible to measure directly without specialized laboratory equipment.
Currently, researchers using Discrete Element Method (DEM) simulators often manually "tune" these parameters through tedious trial and error until the simulation visually matches reality. This research introduces a framework to automate this process, allowing a robot to observe a grain formation, infer the underlying physics, and use that knowledge to perform complex manipulation tasks like pouring without spilling.
The Calibration Framework
The proposed system combines three primary components into a single pipeline to bridge the gap between real-world observation and simulation accuracy.
- Simulation: The framework utilizes NVIDIA’s Isaac Simulator, which employs a GPU-optimized solver to handle thousands of particles simultaneously in real-time.
- Inference (BayesSim): Because simulators are complex "black boxes" where mathematical likelihoods are difficult to calculate, the system uses likelihood-free Bayesian inference. By running thousands of simulations with varied parameters, the system learns a mapping between pile shapes and physical properties.
- Summary Statistics: Rather than comparing raw, noisy images, the system extracts 16 key statistics from depth images, including maximum pile height, average radial distance of grains from the center, and chi-distribution fit.
The workflow is efficient: the system simulates 1,000 different piles to learn the relationship between shapes and physics, then takes a single real-world depth image of a grain pile to estimate the most likely friction and restitution coefficients.
Experimental Setup and Materials
The framework was validated using both synthetic and real-world data:
- Synthetic Data: 1,000 simulated grain formations were used to train the BayesSim model.
- Real-World Materials: Testing was conducted on Israeli couscous (roughly spherical) and barley (ellipsoidal).
- Hardware: Observations were made using an Intel RealSense D435 depth camera, and robotic tasks were executed by an ABB YuMi dual-arm industrial robot.
Results and Generalization
A key insight of this work is "Macro-to-Micro Inference." The macro shape of a pile contains sufficient information to back-calculate micro properties like rolling friction. Notably, even though barley is ellipsoidal and the simulator uses spheres, the inference engine compensates by finding "effective" friction values for spheres that mimic the behavior of non-spherical real-world grains.
Key results include:
- High Precision: In sim-to-sim tests, the system estimated sliding friction with an error of only 0.0062.
- Generalization: Once calibrated on a simple 12cm pour, the simulator accurately predicted the shape of piles poured from varying heights (2cm to 10cm).
- Robotic Task Success: The robot successfully created complex target patterns, such as three concentric rings, by inferring the exact heights needed for pouring. Furthermore, the calibrated simulator accurately predicted the number of grains that would spill during a pouring task, matching experimental results with high fidelity.
Conclusion and Future Work
While the current model assumes identical spherical grains, it effectively compensates for irregular shapes via adjusted friction values. Future work may explore using video sequences to capture the dynamic flow of grains rather than relying on static "steady-state" images. Additionally, incorporating physics for cohesion, moisture, and air drag would expand the framework's utility to a broader range of industrial and culinary materials.
Summary updated