Soft, slow materials such as foam rods, cables, tubing and surgical thread do not jump into a new shape when pushed. They creep toward it, and when released they recoil gradually. The simplest physical description of this behaviour is the Kelvin-Voigt model: a spring and a damper in parallel. Under a steady load, it predicts that deformation approaches a resting shape exponentially, with a single time constant. Learned forecasters for such objects currently regress continuous coordinates frame by frame. That creates two problems. When the object could settle into several distinct shapes (a compressed rod can buckle left or right), a model trained to minimise squared error predicts the average of those shapes, which is physically invalid. Over long holds, small per-frame errors also accumulate into drift.
We propose to split the prediction into two parts. The first is where each segment of the rope will eventually settle. This part can be ambiguous and multimodal, so we predict it as discrete tokens, the way language models predict words. The second is how the rope gets there. Given a resting target and a time constant, the Kelvin-Voigt model has a closed-form answer, so this part is computed exactly rather than learned. A sequence model therefore emits a short set of "equilibrium tokens" and "lag tokens" only when the load or contact changes. The full creep trajectory between those events follows analytically, with no ODE integration and no per-frame decoding.
Core Claim
A deformable linear object (a rope-like object, DLO) can be represented as a polyline in a rotation-invariant local frame: segment lengths plus the bend angle at each joint. We treat each joint angle $\theta_i$ as its own Kelvin-Voigt element. If the load is held constant over an interval, the Kelvin-Voigt law $\sigma = E\varepsilon + \eta\dot\varepsilon$ integrates exactly, which gives
$$\theta_i(t) = \theta_i^{\infty} + \big(\theta_i(t_0) - \theta_i^{\infty}\big)\, e^{-(t-t_0)/\tau_i},$$
where $\theta_i^{\infty}$ is the angle the joint would settle to under the current load, $\tau_i = \eta_i/E_i$ is its retardation time, and $t_0$ is the time of the last load or contact change. The learned model predicts only $\theta_i^{\infty}$, quantized into angle bins, and $\tau_i$, quantized into log-spaced bins. It does this once per event, conditioned on the current polyline tokens and the action.
The claim is that this event-level, physics-structured tokenization does better than two families of models. The first family regresses continuous states (neural ODEs, latent linear dynamics). The second tokenizes raw coordinates every frame, Pix2Seq-style. We expect the gains in three places: mode validity at bifurcations, long-hold extrapolation, and decoding cost. We also expect the fitted $\tau_i$ values to be interpretable material properties.
Relation to Prior Work
The tokenization machinery is borrowed. A Unified Sequence Interface for Vision Tasks quantizes polygon vertices into location tokens and trains with cross-entropy alone. PolyGen does the same for mesh vertices using 8-bit bins. Geometry Informed Tokenization of Molecules shows that tokenizing in a canonical local frame, with distances and angles instead of Cartesian coordinates, removes sensitivity to rotation. We adopt that local-frame idea for polylines. All three works model static snapshots. None of them gives the token stream any temporal or constitutive meaning, and that is the part we add.
The polyline view of ropes comes from Learning Deformable Linear Object Dynamics From a Single Trajectory (NPRBA), which models rods as chains of pseudo-rigid bodies with learned joint forces inside a neural ODE. It is our strongest physical competitor. We keep its segment-chain geometry but replace numerical integration with the closed-form Kelvin-Voigt step. We also replace its deterministic regression with a categorical distribution over resting shapes. Deformable Linear Object Prediction Using Locally Linear Latent Dynamics takes the opposite approach, a black-box latent linear model. It motivates our emphasis on interpretability, since its latent space does not separate stiffness from damping.
On the viscoelastic side, CATCH-FORM-ACTer has a transformer output parallel stiffness and damping gains. Those gains drive a robot's compliance controller; they do not describe the object's own shape evolution. Realistic Surgical Simulation from Monocular Videos deliberately uses a series (Maxwell) model to capture instantaneous deformation followed by relaxation. This work constrains ours. A pure Kelvin-Voigt element has no instantaneous response, and real foams may have one. We test for this directly (see below) instead of assuming it away.
Research Program
The polyline tokenizer fixes the first segment's pose as the reference frame and encodes each subsequent joint by its bend angle. Segment lengths are either learned once per object, as in NPRBA, or fixed. The model itself is a decoder-only transformer. Its input is the current polyline tokens, gripper action tokens, and elapsed time since the last event. At each event it outputs one equilibrium-angle token and one lag token per joint, and between events the closed-form update produces the trajectory.
Training uses cross-entropy on targets obtained by fitting the exponential form to each inter-event window of ground-truth data. As an alternative, we train end to end through the closed-form update, using a straight-through softmax expectation over bins. To address the known weakness that cross-entropy ignores distance between neighbouring bins, we add label smoothing over adjacent bins. We also run an ablation that adds a continuous sub-bin residual.
Data come from three sources. The first is a synthetic benchmark we generate: chains of Kelvin-Voigt joints under compressive and bending loads, with parameters randomised so that buckling direction depends on noise too small to observe. Ground-truth resting shapes are therefore genuinely bimodal, and we know the true $\tau_i$. The second is the aluminum-rod and foam-cylinder trajectories from NPRBA. The third is the rope-manipulation data used by the locally linear latent model. As an optional extension, the synthetic generator adds a series spring (a standard linear solid), which tells us when the pure Kelvin-Voigt assumption fails.
Baselines
The naive reference is a per-joint Kelvin-Voigt chain whose $E_i$ and $\eta_i$ are fit by least squares on the training trajectories. Its resting shapes come from static equilibrium of that fitted chain, with no learned model. If our method cannot beat pure physics with fitted constants, the learned component adds nothing. The comparison uses the same segment count and the same training trajectories.
The strongest existing method is NPRBA (Learning Deformable Linear Object Dynamics From a Single Trajectory), which reports a position RMSE below 5 cm on the foam cylinder using 5 pseudo-rigid bodies trained on one 30-second trajectory. We run it with the same number of segments, the same single-trajectory training regime, and the same hyperparameter search budget as our model. Beating it means lower RMSE on held-out foam trajectories, and in particular a smaller growth of RMSE with horizon length. On the rope dataset, the locally linear latent model reports a 10-step MSE state score of 0.59 ± 0.27 (Deformable Linear Object Prediction Using Locally Linear Latent Dynamics). We rerun it under its published protocol and compare on the same horizon.
Three ablations isolate the key ingredients. First, a Pix2Seq-style per-frame tokenizer uses the same transformer and vocabulary but decodes every timestep's joint angles directly, with no Kelvin-Voigt structure. This isolates the value of the closed-form creep. Second, the same event-level Kelvin-Voigt model is given a continuous head: a Gaussian mixture density head over $\theta_i^{\infty}$ and $\log\tau_i$, with matched parameter count. This isolates discrete tokens against a continuous multimodal head. Third, a plain squared-error regression head is used, which exposes the averaging failure. All models share data splits, compute, and tuning budget.
Why not just…?
Why not just use a neural ODE with Kelvin-Voigt joints? A neural ODE with Kelvin-Voigt joints is NPRBA with a different joint law. It keeps continuous integration, which the NPRBA authors note slows training. It is also deterministic, so on our bifurcating synthetic set it can only predict one resting shape or the average of the two. The synthetic buckling experiment shows this directly by measuring how often predictions land between the valid modes. On unimodal real data it may tie with our method, and we will report that outcome honestly.
Why not just tokenize raw coordinates every frame, as Pix2Seq does? Per-frame tokenization handles multimodality but spends one decoding step per frame. It also has no mechanism that prevents drift during a long hold. The first ablation above is exactly this model. The long-hold extrapolation test (train on short holds, test on holds three to five times longer) is where we expect it to fail. If it does not fail, the Kelvin-Voigt structure is not earning its place.
Why not just put a mixture density head on the same event-level model? A mixture density head is the real threat. A Gaussian mixture can also represent a left-or-right buckle without discrete bins. If it matches the tokenized version on mode validity and accuracy, then the contribution is the event-level Kelvin-Voigt decomposition, not tokenization. We would state that plainly. Tokens may still win on three counts. They make no parametric assumption about the shape of each mode. They yield $\tau_i$ distributions that stay sharp in log space. And they use the same language-model interface as CAD-GPT, where discrete spatial tokens cut invalid outputs to a 1.61% invalidity ratio. That CAD result does not transfer automatically to dynamics, which is why this head-to-head comparison is decisive.
Proposed Experiments
The first decisive check is on the synthetic buckling set. We score each model by the fraction of samples whose resting shape lies within a tolerance of a true mode, and by the error in recovering $\tau_i$ against the known ground truth. We expect squared-error regression to fall between the modes. Tokens and the mixture density head should both cover the modes; the open question is whether tokens do so more reliably at small bin counts, and we sweep 32 to 512 bins to map the precision-versus-vocabulary trade-off.
The second check is long-hold extrapolation on the foam cylinder and the synthetic set. We plot RMSE against horizon for our model, NPRBA, and the per-frame tokenizer. The prediction is that closed-form creep keeps error flat, while the other two grow.
The third check tests whether Kelvin-Voigt alone is sufficient. On standard-linear-solid synthetic data and on foam, we look for a systematic residual immediately after each event. Such a residual would reveal a missing instantaneous response of the kind SurgiSim's Maxwell choice anticipates. If we find it, we add one instantaneous-jump token per joint and report how much it helps.
Finally, we report decoding steps and wall-clock time per simulated second against the per-frame tokenizer and NPRBA. This measures whether event-level decoding actually removes the latency cost that autoregressive spatial models usually carry.
References
- A Unified Sequence Interface for Vision Tasks
- PolyGen: An Autoregressive Generative Model of 3D Meshes
- Geometry Informed Tokenization of Molecules for Language Model Generation
- Learning Deformable Linear Object Dynamics From a Single Trajectory
- Deformable Linear Object Prediction Using Locally Linear Latent Dynamics
- CATCH-FORM-ACTer: Compliance-Aware Tactile Control and Hybrid Deformation Regulation-Based Action Transformer for Viscoelastic Object Manipulation
- Realistic Surgical Simulation from Monocular Videos
- CAD-GPT: Synthesising CAD Construction Sequence with Spatial Reasoning-Enhanced Multimodal LLMs