Evolving the Time Constants of Touch

Most learned tactile models quietly treat each moment of contact as if it were settled. A static tactile image shows how deeply a soft fingertip is pressed in, but it cannot say how quickly the material relaxes, whether a grasp is slowly starting to rotate, or how firm an object feels after two seconds compared with the first fifty milliseconds. This is the same mistake that steady-state models make in engineering, where setting time derivatives to zero makes a problem easy to solve but hides lag, overshoot and short spikes. We propose a small, fast temporal module for tactile perception. It sits on top of a frozen pretrained touch encoder and represents contact as a sum of relaxation processes, each decaying at its own rate. The rates themselves, which are the few numbers gradient descent handles worst, are found by evolutionary search before ordinary training begins.

The two-sentence version: viscoelastic contact and slip unfold on time scales from milliseconds to seconds, so a tactile model needs a well-chosen set of "memory time constants." Gradients are poorly suited to finding those constants. Evolutionary search is well suited to a space of a few dozen numbers. If the proposal is right, a model of roughly a thousand parameters on top of a frozen encoder should match or beat heavy video-transformer and recurrent tactile models on dynamic tasks, while running fast enough for reactive control. The chosen time constants should also line up with measurable physical relaxation times.

Core Claim

The open problem in evolutionary weight initialization is scale. Population-based search breaks down when asked to encode millions of weights. The demonstrated successes are small: a five-input radial-basis glove classifier in GK-RBF, and black-box problems of up to 640 dimensions in EvoGrad.

The open problem in temporal touch is latency. Neural Contact Fields needs a ConvLSTM taking about 66 ms per forward pass. The 16-frame VideoMAE (a video transformer pretrained by reconstructing masked patches) in DIGIT Pinki and the frame-difference Video-MAE in AnyTouch 2 both need multi-frame buffers that add delay to the control loop.

Both problems shrink once we observe that the hard-to-learn part of a temporal tactile model is low-dimensional. That part is the spectrum of time scales the model can remember, not the spatial features, which self-supervised encoders such as Sparsh already supply well. We claim three things:

  1. Expressing tactile memory as a bank of first-order relaxation modes turns the initialization question into a search over roughly 10–40 numbers.
  2. Gradient descent places those numbers poorly, because the loss is exponentially insensitive to long time constants.
  3. A short evolutionary search over the distribution of time constants, followed by ordinary gradient training, gives better dynamic tactile predictions than the same head with default or grid-spaced constants, at matched compute.

The Model

Let $z_t$ be the frozen encoder's feature vector for tactile frame $t$, and let $\Delta t$ be the frame interval. The head keeps $K$ hidden states. Each state $h_k$ is a leaky integrator with time constant $\tau_k$:

$$ h_k(t) = e^{-\Delta t/\tau_k}\, h_k(t-1) + \left(1 - e^{-\Delta t/\tau_k}\right) W_k z_t , $$

Here $W_k$ is a small learned projection. A linear or two-layer readout maps $[z_t, h_1(t), \dots, h_K(t)]$ to the task output: slip, stability, force, or hardness.

This is a learned version of the Prony series, the classical sum-of-exponentials description of viscoelastic stress relaxation. The term $z_t$ alone plays the role of the "steady-state" reading. Each $h_k$ carries one transient mode. One update costs $K$ multiply-adds per projected feature and needs no frame buffer. Latency is therefore fixed by the encoder, not by the temporal model.

The difficulty is visible in the formula. The gradient of the loss with respect to $\log \tau_k$ passes through $e^{-\Delta t/\tau_k}$, which is nearly constant whenever $\tau_k \gg \Delta t$. Long memories therefore get almost no gradient signal. Constants that start in the wrong range stay there.

Research Program

We do not evolve individual weights. Following Sun et al., who evolved the mean and variance of Gaussian weight initializers rather than the weights themselves, we evolve the parameters of a small mixture distribution over $\log \tau$. Concretely, we search over $M \le 4$ components with means, spreads and weights, together with the mode count $K$ and the initial scale of the projections $W_k$. The search space has under 20 dimensions.

The search uses CMA-ES (Covariance Matrix Adaptation Evolution Strategy, which adapts a Gaussian over candidate solutions from the scores of a population). Each candidate's fitness is the validation loss after a short, fixed gradient budget, for example 200 Adam steps on the frozen-encoder features. Encoder features are computed once and cached, so each fitness evaluation costs seconds rather than a GPU-day. This is what makes evolutionary search affordable here. As a second variant we run EvoGrad2's combination of evolutionary sampling with learned gradients and Hessian corrections, since it solved more COCO functions than CMA-ES at a smaller evaluation budget.

After the search, the best candidate is trained to convergence with ordinary gradient descent. The time constants remain trainable in this phase, so the evolutionary step really is an initialization.

The steady-state lens enters in two ways. As an ablation, we remove all transient modes and keep only $z_t$, which measures how much error a quasi-static reading leaves. As an analysis, we check whether evolved time constants cluster near the physical relaxation times that Burgeson et al. observed, where apparent firmness changes by orders of magnitude within two seconds of contact, and near the much faster slip time scales targeted by AnyTouch 2's frame-difference objective. This addresses the tension the file identifies between Sparsh, which argues that latent objectives avoid sensor-specific pixel artifacts, and AnyTouch 2, which argues that pixel-level frame differences are needed to catch incipient slip. If explicit fast modes over latent features recover slip sensitivity, the transient information does not have to come from pixel reconstruction.

Baselines

The naive reference is the frozen Sparsh encoder with a per-frame readout and no temporal state. This is the steady-state reading. Next to it is the same relaxation head with time constants spaced log-uniformly between one frame and the sequence length. That grid is the default a careful engineer would choose, and it isolates the proposal's key ingredient: searched time constants versus sensible fixed ones.

The second ablation spends the same compute in other ways. It trains that head from many random seeds and keeps the best on validation. It also tunes the time constants by random search and by a Bayesian hyperparameter optimizer, with the identical number of fitness evaluations and identical inner-loop budget. The file reports that hyperparameter optimization with Optuna improved MLP accuracy by 12–17% in Heat Exchanger Surrogates, so this is a serious competitor rather than a straw man.

A third baseline is a gated recurrent head (an LSTM) of matched parameter count on the same frozen features. This tests whether the relaxation structure matters or whether any recurrence suffices.

The strongest task-specific competitors are the published sequential models.

  • PoseIt. On PoseIt, a sequential ResNet50+LSTM with deferred resampling reports 85.2% grasp-stability accuracy on unseen holding poses.
  • Taxim real grasps. On the real-robot grasp set from Grasp Stability Prediction with Sim-to-Real Transfer, a ResNet-18+LSTM reports 90.7% zero-shot accuracy.
  • Force regression. For force, Sparsh reports 3-axis errors below 0.1 N.
  • Slip. For TacBench slip detection and ToucHD dynamic tasks, no reported result was found on Lacuna for a numeric slip or ToucHD baseline. There we will retrain AnyTouch 2's released model under our protocol.

All methods see the same training splits. Where we retrain baselines, they receive the same total GPU-hours, with the evolutionary search counted inside our budget. Beating the evolved head's own ablations means a statistically reliable gain over the log-uniform grid and over equal-budget random or Bayesian search, across at least five outer seeds. Beating the published systems means matching 85.2% on PoseIt unseen poses, or exceeding it, at a fraction of the parameters and with lower per-step latency.

Why not just…?

Why not just fine-tune a tactile video transformer end to end? AnyTouch 2 and DIGIT Pinki show this works for accuracy: 98% on 44 silicone hardness classes for the latter. Both need multi-frame windows, though, and the file lists that phase lag as an open problem for reactive control. We include an AnyTouch 2 fine-tune as an accuracy ceiling and report latency next to it. If it wins on accuracy, our contribution narrows to the accuracy–latency trade-off, and we will say so.

Why not just space the time constants on a log grid and skip the search? This is the real threat. It may work as well whenever the task's time scales fall comfortably inside the grid. That is exactly why the log-grid head is our central ablation. The proposal predicts the gain will concentrate where relevant time scales are clustered or extreme: millisecond slip onset in the PoseIt shake phase, and second-scale relaxation in hardness tasks. If the grid ties the evolved head everywhere, the conclusion is that structure matters and the search does not. That is still a publishable negative result about evolutionary initialization.

Why not just use self-supervised pretraining, which already gives good starting weights? We do use it. Sparsh's 95.1% average gain over end-to-end models in low-label regimes is why the encoder is frozen and borrowed. Pretraining on images or short clips says nothing about which memory spans a downstream task needs. The evolutionary step targets only that gap.

Why not just run Optuna instead of CMA-ES? At under 20 dimensions, Bayesian optimization is a fair competitor. We run it at equal budget and claim no advantage for evolution beyond what that comparison shows. EvoGrad2's higher solve rate than CMA-ES on COCO suggests the search method matters, but that evidence comes from synthetic functions, not this landscape.

Proposed Experiments

Primary benchmarks. We use PoseIt for transient grasp stability, scored as accuracy on unseen poses with the shake phase reported separately, since instability rose by 4% during shaking there. We add the TacBench slip-detection and grasp-stability tasks from Sparsh, reported at 1%, 10% and full labels, and the force-paired tier of ToucHD, scored as RMSE on force change.

Latency. We measure per-frame latency on a single consumer GPU and on CPU, alongside Neural Contact Fields' reported 66 ms as a reference point.

First decisive check. At equal compute, evolved initialization must beat both the log-uniform grid and equal-budget random and Bayesian search on at least the transient-heavy tasks: the PoseIt shake phase and slip.

Second decisive check. The no-transient (steady-state) ablation must lose substantially on these tasks while staying competitive on near-static force estimation. That pattern would show the head is capturing genuine dynamics rather than adding capacity.

Third decisive check. Time-constant distributions evolved on one task or sensor must transfer to another. For example, constants evolved on Taxim-simulated grasps should transfer to the real GelSight set, and constants from ToucHD should transfer to PoseIt. Transfer would show the search finds physical time scales of contact rather than overfitting a validation set.

Physical-consistency check. We fit a classical Prony series to raw force-relaxation traces in ToucHD's force tier and compare its relaxation times with the evolved $\tau_k$.