When a robotic arm interacts with the human world, it immediately encounters a messy, multi-material reality. Preparing a meal might require a robot to slice a solid vegetable, pour a viscous sauce, and handle a flexible cloth wipe.
Historically, simulating these environments meant combining entirely separate physics engines: a rigid-body solver for the robot, a Navier-Stokes grid for the liquid, and a mass-spring or thin-shell system for the cloth. As noted by Yi-Ling Qiao et al. in Scalable Differentiable Physics for Learning and Control and Siwei Chen et al. in DaxBench: Benchmarking Deformable Object Manipulation with Differentiable Physics, this fragmentation forces researchers into siloed tools. More importantly, when these distinct solvers try to talk to one another at the boundaries—such as a fluid pressing against a deformable fabric—the math often breaks down, leading to numerical instability, fluid leaks, or artificial energy explosions.
The research direction of unified multi-material interaction modeling solves this boundary problem by tearing down the walls between material types. Instead of treating solids, liquids, and fabrics as fundamentally different mathematical entities, it discretizes everything into a single, shared primitive: the particle.
The Intuition: Everything is a Particle
In a unified particle-based framework, the macroscopic difference between water, a rubber ball, and a steel rod is not what they are made of, but how their constituent particles interact. By representing diverse states of matter under a single geometric abstraction, the nightmare of calculating complex boundary conditions between different solvers disappears. A fluid particle colliding with a rigid body particle is handled by the exact same proximity and collision laws as two fluid particles colliding.
As argued by Yunzhu Li et al. in Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids, particles provide a universal inductive bias that allows a single computational pipeline to govern an entire multi-material scene. Under this paradigm, the physical state of an object is dictated entirely by varying the connectivity rules and internal constraints between its particles:
- Rigid Bodies: Modeled by enforcing fully connected, unyielding internal constraints that force the particles to maintain their relative distances perfectly.
- Deformable Solids: Modeled using sparse, flexible connections or elastic potential energies that allow particles to deflect and snap back.
- Fluids and Granular Matter: Modeled with weak or transient constraints governed by pressure, viscosity, and friction.
For example, Sinuo Liu et al. demonstrate in A unified framework for packing deformable and non-deformable subcellular structures in crowded cryo-electron tomogram simulation that by simply altering these internal connectivity rules under a shared physics engine, one can simultaneously simulate the rigid collisions of macromolecules alongside the complex deformations of cellular membranes and filaments.
Tuning the Material Continuum
To make a single particle solver capture the full spectrum of physical behaviors—including complex non-Newtonian materials that act like liquids under some forces and solids under others—the underlying mechanics must decouple and blend different physical stresses.
In A Unified Particle-Based Solver for Non-Newtonian Behaviors Simulation, Chunlei Li et al. show that total material stress can be mathematically split into independent elasto-plastic and viscous components. By dynamically adjusting the balance between these components, a single solver can transition seamlessly from simulating shear-thinning liquids to shear-thickening solids. This matches the classic graphics insight from James F. O’Brien in Animating Sand, Mud, and Snow, which proved that adjusting a small set of parameters like compression and roughness within a single height-field framework could capture the distinct behaviors of highly diverse granular terrains.
[Particle Representation]
│
├─► Strict Internal Constraints ──► Rigid Solids
├─► Elastic/Plastic Potentials ──► Deformable Bodies
└─► Viscous & Pressure Forces ──► Fluids / Gases
This continuum approach is particularly powerful for data-driven and neural physics engines. Instead of hand-coding differential equations for every material phase, a neural network can learn the underlying physics directly from particle graphs. As shown by Alvaro Sanchez-Gonzalez et al. in Learning to Simulate Complex Physics with Graph Networks and Damian Mrowca et al. in Flexible neural representation for physics prediction, embedding material-specific properties (like stiffness or viscosity) directly as features within particle nodes or relation vectors allows a single graph neural network to predict the evolution of fluids, sand, and rigid bodies simultaneously.
Eliminating Leaks and Interpenetration
While treating everything as particles simplifies interaction math, it introduces a major vulnerability: numerical penetration. If a fluid particle travels too fast, or if a solid boundary is too thin (like a sheet of cloth), particles can slip past each other, causing unphysical "leaks."
To guarantee physical integrity during contact-rich interactions, modern unified frameworks incorporate advanced contact mechanics directly into the particle optimization loop. In Intersection-free Robot Manipulation with Soft-Rigid Coupled Incremental Potential Contact, Wenxin Du et al. utilize the Incremental Potential Contact (IPC) method to treat rigid and soft bodies under a single mathematical optimization problem. This ensures completely intersection-free manipulation and consistent frictional forces.
When dealing with solid-fluid coupling across different geometric dimensions (such as a 3D fluid interacting with a 1D thin rod or a 2D shell), Minchen Li et al. introduce a solution in A Contact Proxy Splitting Method for Lagrangian Solid-Fluid Coupling. By utilizing a "contact proxy" to resolve coupling forces in a separate phase, their method guarantees leak-proof boundaries without sacrificing the speed of the underlying Lagrangian framework.
Hybrid Solvers and Differentiable Control
Pure particle methods can sometimes be computationally expensive when scaled to massive scenes. To balance efficiency with multi-material expressiveness, researchers frequently deploy hybrid architectures or neural emulators.
For instance, Haoran Lu et al. in GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation combine Position-Based Dynamics (PBD) for large-scale fabric and fluid interactions with the Finite Element Method (FEM) for highly elastic items. In the neural domain, Omer Rochman Sharabi demonstrates in A Neural Material Point Method for Particle-based Emulation that a hybrid Lagrangian-Eulerian approach (NeuralMPM) can map particle states onto a voxelized Eulerian grid. This architecture leverages the flexibility of particles for material tracking while utilizing the grid for highly efficient, image-like collision handling via standard neural architectures.
Ultimately, the true power of unifying these physical modalities under a single computational graph is that it allows the entire system to be differentiable. When the simulation can compute analytical gradients across heterogeneous material boundaries, robotic agents can use gradient-based optimization to learn complex control policies natively.
As shown in frameworks like SoftZoo: A Soft Robot Co-design Benchmark For Locomotion In Diverse Environments by Tsun-Hsuan Wang et al. and DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation by Zilin Si et al., a robot can optimize its movements, morphology, and tactile feedback simultaneously, whether it is navigating through mud, grasping a rigid tool, or squeezing a soft sensor against a deformable object.
Key Foundational Work
A material point method for snow simulation (2013) — contributed 2 concept elements to this direction.