Robots trained in simulation usually treat the objects they touch as uniform blocks of rubber or plastic. Many real objects are not uniform. Fiber-reinforced polymers, rubberized panels and laminated parts are composites: they mix materials so that different ingredients carry load at different stages of deformation. A fine fiber stiffens the part at small strain, and a coarse fiber takes over at larger strain. Two parts can feel identical under a light touch and behave very differently under a firm press or a sustained hold. We propose a small, differentiable contact law that describes this staged stiffening and slow relaxation with a handful of parameters. The law is placed inside a tactile simulator and identified from a few seconds of deliberate probing with a touch sensor. The claim is that this staged structure, which a single "effective stiffness" cannot represent, is what breaks tactile sim-to-real transfer on composites, and that a short, well-designed probing motion recovers it.
The test is designed to fail cleanly if the claim is wrong. We fabricate specimen pairs that share the same small-strain stiffness but differ in how their reinforcement is arranged. A simulator that sees only effective stiffness cannot tell the members of a pair apart. If a policy trained with our law beats both a homogenized model and a history-conditioned randomized policy on these pairs, the composite structure matters. If it does not, we will have shown that a simpler temporal summary is enough, which is also a useful result.
Core Claim
The composites literature shows that hybrid reinforcement gives nonlinear, stage-wise behavior that simple mixture rules do not predict. Research on mechanical properties and prediction methods of hybrid fiber concrete for airport pavements describes fine fibers arresting micro-cracks early in loading and coarse fibers handling macro-cracks later, and it needs an empirical hybrid effect coefficient to capture the combination. Enhanced Dynamic Compressive Behavior of Rubberized Concrete with Steel–Glass Fibers similarly splits roles: rubber absorbs energy and fibers arrest cracks. These papers motivate the proposal but stay at the scale of structural testing. None of them asks what a fingertip sensor feels.
Our claim is that, at the scale of robot manipulation, this staged behavior shows up in the time series of contact force and sensor-gel deformation. It therefore follows a characteristic pattern that can be identified from a short, rate-varied probe. We model the contact stress with a reduced law:
$$ \sigma(\varepsilon, t) = E_0\,\varepsilon + \sum_{k=1}^{2} E_k\,\mathrm{softplus}\!\big(\beta(\varepsilon-\varepsilon_k)\big)/\beta \;-\; \int_0^t \sum_{j=1}^{2} g_j\, e^{-(t-s)/\tau_j}\,\dot\sigma_e(s)\,ds $$
The symbols are defined as follows:
- $\sigma$ is the contact-normal stress.
- $\varepsilon$ is local compressive strain.
- $E_0$ is the matrix stiffness.
- $E_k$ and $\varepsilon_k$ are the added stiffness and engagement strain of the fine ($k=1$) and coarse ($k=2$) reinforcement.
- $\beta$ is a fixed smoothness constant that keeps the law differentiable.
- $\sigma_e$ is the instantaneous elastic part of the stress, given by the first two terms.
- $g_j$ and $\tau_j$ are the magnitudes and time constants of two relaxation modes.
The law has eight parameters, which is few enough to identify from seconds of data. It still separates the two effects a homogenized model merges: stiffening that turns on with strain, and softening that depends on time. The softplus form is a smooth version of the piecewise stiffening described qualitatively in the hybrid-fiber work. The two relaxation modes follow the observation in FORTE that viscoelastic drift needs several time windows (2.5, 5 and 10 s) to be corrected.
Research Program
The simulator. The method builds on DIFFTACTILE, a differentiable tactile simulator. It models a soft sensor gel with finite elements and manipulated objects with a particle-based continuum method (MLS-MPM), and it calibrates stiffness and friction by pushing real sensor errors back through the simulation using gradients. DIFFTACTILE supplies the method, but it models objects only as uniform elastic or elastoplastic media. Our change is to replace the object material law with the reduced composite law above.
Two design choices follow from this:
- Anisotropy. The law acts per particle, so fiber direction can be expressed by applying it along a local material axis.
- What we deliberately leave out. We do not model individual fibers, delamination or crack growth. That would be computationally prohibitive, and the direction overview already flags it as an open problem. The bet is that a manipulating robot needs only the staged, time-dependent stress seen at the surface.
The temporal interaction piece. The identification signal comes from an interaction sequence rather than a single snapshot. A static contact map of the kind produced by TOUCH or CHOIR cannot separate strain-activated stiffening from time-dependent relaxation. Both papers generate isolated contact configurations with no force or time dimension. We instead design a short probe with five phases:
- A slow indentation ramp to moderate depth.
- A fast ramp to the same depth.
- A hold.
- A deeper ramp.
- A small lateral slide.
The phases are chosen so that each parameter group dominates somewhere:
- The fast/slow comparison isolates the relaxation time constants $\tau_j$.
- The two depths isolate the engagement strains $\varepsilon_k$.
- The slide exposes friction and the onset of stick-slip.
Stick-slip matters because Grasp Stability Prediction with Sim-to-Real Transfer from Tactile Sensing found that friction miscalibration dominates transfer error. We will also optimize the probe itself, by choosing ramp depths and rates that maximize the determinant of the Fisher information of the law's parameters under the simulator. This tests whether a designed probe beats a generic press.
After identification. The identified parameters are used in two ways:
- They condition a policy trained in the calibrated simulator.
- They are used to generate synthetic tactile sequences for a grasp-stability predictor, using the ResNet-18 + LSTM temporal architecture of Si et al.
Baselines
Naive reference. The do-nothing reference is DIFFTACTILE exactly as published: objects modeled as homogeneous elastic media, with stiffness and friction calibrated by its gradient-based identification on the same probe data. It is the natural starting point because it already beats reinforcement learning without gradients by a wide margin. DIFFTACTILE reports a success score of 60.82 against 4.68 for SAC on block repositioning, within the same training budget (DIFFTACTILE). On composite specimens there is no reported result found on Lacuna for DIFFTACTILE or any tactile simulator. The comparison is therefore run by us, under the following fairness conditions:
- identical probe trajectories,
- identical numbers of optimizer steps,
- identical policy training budgets.
Strongest existing transfer result. The strongest existing tactile sim-to-real result in the file is the calibrated Taxim pipeline of Grasp Stability Prediction with Sim-to-Real Transfer from Tactile Sensing. It reports 90.7% zero-shot real grasp-stability accuracy on 12 objects, against 42.6% for the TACTO simulator. That pipeline maps normal force linearly to indentation, so it is a homogenized model by construction. We rerun it on our specimens, using the same GelSight sensor and the same UR5e arm used in both prior papers, and calibrate its friction on the same probe data. We would count as beating it if we outperform it on the matched-modulus composite specimens while matching it (within noise) on the monolithic controls. This matters because a method that only helps where the baseline was already weak is less convincing than one that leaves easy cases untouched.
Strongest simple competitor. The strongest simple competitor is a history-conditioned randomized policy. It uses the homogenized simulator but with an added single-mode viscoelastic relaxation, trains across wide randomized ranges of stiffness, friction and relaxation time, and feeds the policy a window of past tactile frames so that it can adapt implicitly. It receives the same simulated sample budget and the same hyperparameter search budget (an equal number of trials) as our method.
Ablation. The key ablation removes the staged stiffening terms ($E_1 = E_2 = 0$) and keeps the relaxation terms. This isolates whether strain-activated reinforcement adds anything beyond viscoelasticity.
Real-only reference. We also include a real-only tactile imitation policy in the style of DECO, trained on the same total robot time spent collecting demonstrations. DECO shows that tactile input lifts dual-arm success from 72.25% (vision-only) to 82.50%, but it has no simulator. This reference therefore measures what simulation contributes at all.
Why not just…?
Why not just randomize a homogenized simulator and let a history-conditioned policy adapt? This is the most serious threat, and we treat it as a hypothesis rather than a straw man.
- What the prior evidence covers. Wide search without accurate contact mechanics has done poorly: SAC scored 4.68 against 60.82 in DIFFTACTILE, and uncalibrated TACTO reached 42.6% against 90.7% in Si et al. But those comparisons did not give the randomized policy a memory of past tactile frames.
- Why it could still fail here. Randomizing a single stiffness over a range only covers composites whose behavior lies inside the homogenized family. A matched-modulus specimen that suddenly stiffens at 8% strain lies outside that family at any randomization width.
- How the experiments settle it. The deciding experiment is the matched-pair evaluation below. If the history-conditioned policy closes the gap there, we will report that explicit composite modeling is unnecessary.
Why not just measure an effective modulus for each object and use it? Homogenization assumes a linear mixture rule. The hybrid-fiber literature (Research on mechanical properties and prediction methods of hybrid fiber concrete for airport pavements) needed a nonlinear hybrid effect coefficient precisely because such rules fail. Our specimen design makes this concrete: matched pairs share the effective modulus by construction, so any performance difference within a pair refutes homogenization for that task. We also report tasks where the pairs do not differ, such as light surface following. Those tasks mark the regime where the simpler model is adequate.
Why not just translate real tactile images into simulator-looking ones? Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation bypasses calibration by mapping real sensor images into the simulated domain. On matched-modulus pairs, however, the small-strain images are nearly identical by design. The translator cannot invent a stiffening response the simulator never produces, so the policy still acts on the wrong dynamics under deep presses. Sim-to-Real Causal Transfer also shows that representations without physical or causal constraints latch onto spurious regularities, which is a further reason for doubt. We include the translation approach as an additional baseline on the grasp-stability task, where it is cheapest to run.
Why not just learn a neural residual on top of DIFFTACTILE's homogenized law? A residual network with enough capacity could, in principle, fit staged stiffening. The concern is whether it can be fit from seconds of data. We compare our eight-parameter law against an MLP residual stress term trained on the same probe data. The comparison measures prediction error on held-out probe rates and depths not seen during identification. We expect our law to extrapolate better, because its form fixes where stiffening can occur. If the residual matches it, the composite-specific form is not needed.
Proposed Experiments
Specimens. We cast silicone blocks and bars in four families:
- monolithic,
- fine-fiber only (short microfibers),
- coarse-fiber only (embedded mesh or rods),
- hybrid.
Fiber fraction is tuned so that each non-monolithic specimen matches a monolithic partner's small-strain modulus, measured on a bench indenter. Specimens are made in two geometries and with two fiber orientations. Bench stress-strain-time curves serve as ground truth for the identified parameters.
Hardware. We use a UR5e with a GelSight sensor, as in DIFFTACTILE and Si et al., so that results are comparable to theirs on monolithic controls.
Tasks. The protocol covers three tasks:
- Grasp-and-lift stability prediction. We follow the Si et al. protocol and report accuracy. Our specimens are added alongside that paper's 12 YCB/GoogleScan objects, and we re-collect data on those objects with our own setup.
- Controlled pressing to a target force during a 10 s hold. We report force-tracking RMSE, with FORTE's 0.187 N on compliant fingers as a reference scale, not as a direct competitor.
- Surface following and block repositioning. These are taken from the DIFFTACTILE task suite, and we report success scores.
Identification quality. We measure it in two ways:
- parameter error against bench ground truth,
- held-out tactile-sequence prediction error, reported both as image PSNR (DIFFTACTILE's optical module reaches 39.42) and as force-curve error on unseen probe rates.
Decisive checks.
- Within-pair gap on hybrid specimens. Our method should shrink the within-pair performance gap on hybrid specimens relative to the homogenized DIFFTACTILE and calibrated-Taxim baselines, while leaving monolithic performance unchanged.
- Whether the stiffening terms carry the gain. The stiffening-free ablation should lose most of that gain on deep-press and lifting tasks, but not on light surface following.
- Whether probe design matters. The information-optimized probe should identify $\varepsilon_k$ and $\tau_j$ with lower error than a single constant-rate press of equal duration.
How we will read the outcomes.
- If the history-conditioned randomized policy matches our method on the first check, the conclusion is that temporal memory, not composite structure, is what transfer needs. We will publish it that way.
- If our law wins only when reinforcement is aligned with the pressing direction, that locates where anisotropy begins to matter for touch.
References
- Research on mechanical properties and prediction methods of hybrid fiber concrete for airport pavements
- Enhanced Dynamic Compressive Behavior of Rubberized Concrete with Steel–Glass Fibers
- FORTE: Tactile Force and Slip Sensing on Compliant Fingers for Delicate Manipulation
- DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation
- TOUCH: Text-guided Controllable Generation of Free-Form Hand-Object Interactions
- A Versatile and Differentiable Hand-Object Interaction Representation
- Grasp Stability Prediction with Sim-to-Real Transfer from Tactile Sensing
- DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter
- Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation
- Sim-to-Real Causal Transfer: A Metric Learning Approach to Causally-Aware Interaction Representations