RFW-S: Everyday Deformable Objects with Independent Simulation Verification

Jiarui Guo, Saining Zhang, Jiaying Wan, Jiawei Weng, Junhao Chen, Rui Zhang, Yuanbo Lyu, Zhiheng Lee, Peishuo Li, Jingrun Du, Weiqing Xiao, Weiyi Lin, Zenan Meng, Hu Wei, Lin Qu, Zongzheng Zhang, Hao Zhao, Chaojian Li

TL;DR

RFW-S (RoboFlyWheel-Soft) is a massive new dataset of over 83,000 simulation-verified deformable assets—ranging from sponges to silk blouses—designed to bridge the gap between visual 3D diversity and physical realism in robotics. By combining LLM-based parameter estimation with a rigorous "independent verification" step, the authors ensure that these objects don't just look right, but behave correctly under varied physical interactions.

What Problem Does This Paper Address?

Robots in the real world must handle objects that squish, fold, and stretch. While we have massive datasets of rigid 3D objects (like Objaverse), deformable assets are notoriously difficult to scale.

The challenge isn't just visual; it’s structural and physical. A 3D mesh of a bag is useless for simulation if the solver doesn't know it has a cavity, or if its material parameters (like stiffness and friction) are just guessed. Furthermore, an object might "pass" a simple drop test during calibration but fail catastrophically when squeezed or lifted. Existing datasets often lack this "executable" physical grounding, making it hard to train robots for complex manipulation tasks like folding laundry or handling fragile food.

Methods

The authors developed an automated pipeline to turn 2D references into simulation-ready assets:

  1. Generation & Adaptation: They use a vision-language model (Qwen-Image-2.0) to generate reference images, which are then converted into 3D meshes. Crucially, they perform structure-aware modeling, adapting the mesh into specific "profiles" (e.g., filled volumes for soft solids, thin sheets for cloth) compatible with physics solvers like Genesis.
  2. LLM-Assisted Parameterization: A multimodal LLM estimates physical parameters (density, Young’s modulus, etc.) based on the object's appearance and category.
  3. The Calibration Loop: The object is put through a "calibration probe" (like a simple drop). If it fails to behave realistically, the LLM receives feedback from the simulator and suggests bounded corrections to the parameters.
  4. Independent Verification: This is the paper's "secret sauce." Once an object passes calibration, its settings are frozen and it is subjected to a different stimulus (e.g., lateral compression). This ensures the object's physical configuration is robust and hasn't just been "overfitted" to the calibration test.
The RFW-S pipeline: from planning and geometry generation to the simulation-guided "Author & Verify" loop.

Datasets

The primary contribution is the RFW-S Dataset:

  • Scale: 83,365 simulation-verified assets.
  • Breadth: 485 everyday categories.
  • Structural Families: Five distinct groups: soft solids, cloth sheets, shells, garments, and bags.
RFW-S covers a vast range of everyday items, categorized into five structural families for robotic interaction.

Key Insights and Ideas

  • Structure > Material: Tuning material parameters (like making a mesh "softer") cannot compensate for a poor structural representation. For example, a garment needs a specific cloth profile to fold naturally; a standard solid mesh won't suffice regardless of its stiffness settings.
  • The Verification Gap: Calibration success does not guarantee general physical validity. The authors found that a small but significant percentage of assets passed calibration but failed independent verification, proving that multi-interaction testing is essential for reliable simulation.
  • Integrity vs. Success: In robotic grasping, "task success" (lifting the object) is different from "integrity." A robot might successfully lift a piece of tofu but crush it in the process. RFW-S includes metrics to evaluate this distinction.

Key Results

  • High Retention: Out of an initial pool of 100,000 assets, 83,365 (83.37%) passed both physical verification and behavioral contract checks.
  • Robustness: The independent verification step caught nearly 4% of assets that would have otherwise been accepted despite failing under secondary interactions.
  • Sim-to-Real Realism: The authors demonstrated "fragile-object manipulation" by replaying simulated gripper commands on a real-world Piper robot arm. The real robot reproduced the same outcomes—both successful intact transfers and "crushing" failures—seen in the simulation, validating the dataset's physical accuracy.
The calibration and verification framework ensures that assets are tested against distinct stimuli before being accepted.

Limitations and Future Work

  • Uneven Retention: Some categories, like "soft solids" (e.g., gummy candy, toffee), had lower pass rates than cloth, indicating that current representations for highly viscous or sticky materials still need work.
  • Integrity Metrics: The current "integrity" proxy (based on deformation gradients) can sometimes be triggered by simple rigid rotations. The authors suggest moving toward more complex, rotation-invariant strain measures.
  • Future Directions: The team aims to extend RFW-S to hybrid objects (e.g., a bag with a rigid handle) and use these assets as building blocks for entire generative, physically grounded 3D worlds.

Summary updated

Abstract

Large-scale robotic simulation requires diverse deformable objects with executable physical properties and realistic interaction behaviors. However, existing 3D asset collections primarily emphasize visual diversity, while scalable physical construction and verification remain challenging. We introduce RFW-S, a large-scale dataset of 83,365 simulation-verified assets spanning 485 everyday categories across five structural families: soft solids, cloth sheets, shells, garments, and bags. We develop an automated pipeline integrating reference-guided 3D generation, structure-aware physical modeling, and LLM-assisted parameter estimation and calibration. To ensure physical reliability, we introduce an independent simulation verification framework that evaluates object behaviors under interactions distinct from those used for parameter calibration. Extensive experiments demonstrate the effectiveness of independent verification and characterize deformable responses under different physical configurations. We further evaluate robotic grasping in terms of both task success and object integrity, and demonstrate the physical realism of our simulated objects through sim-to-real replay of fragile-object manipulation. RFW-S provides a scalable foundation for physically grounded simulation and embodied robotic learning.
BibTeX
@misc{guo2026rfw,
  author = {Jiarui Guo and Saining Zhang and Jiaying Wan and Jiawei Weng and Junhao Chen and Rui Zhang and Yuanbo Lyu and Zhiheng Lee and Peishuo Li and Jingrun Du and Weiqing Xiao and Weiyi Lin and Zenan Meng and Hu Wei and Lin Qu and Zongzheng Zhang and Hao Zhao and Chaojian Li},
  title = {{RFW-S: Everyday Deformable Objects with Independent Simulation Verification}},
  year = {2026},
  eprint = {2611.23056},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23056},
}
Figure