Event-based Visual Deformation Measurement

Yuliang Wu, Wei Zhai, Yuxin Cui, Tiesong Zhao, Yang Cao, Zheng-Jun ZhaarXiv 2026 · arXiv

TL;DR

Traditional visual deformation measurement relies on data-heavy high-speed cameras to track complex, rapid surface motion. This paper introduces an event-frame hybrid framework that pairs the high temporal resolution of event cameras with the spatial precision of standard frames. By linearizing deformations over triangular meshes and using a neighborhood-guided optimization strategy, the method achieves a 1.6× higher tracking survival rate under large displacements while slashing data storage and processing costs by over 80%.

What Problem Does This Paper Address?

Visual Deformation Measurement (VDM) is a non-contact technique used to recover dense deformation fields by tracking surface motion. It is vital for structural health monitoring, material mechanics analysis, and biomechanics research.

Unlike rigid body tracking—which only requires estimating 6 degrees of freedom—deformable surfaces have massive degrees of freedom because every surface point can move independently. Traditional image-based methods handle this by using expensive high-speed cameras to ensure minimal motion between consecutive frames, which keeps the correspondence search space manageable. However, this approach creates a massive bottleneck: processing and storing thousands of redundant frames incurs prohibitive computational and storage overhead, severely limiting practical scalability.

Methods

The authors propose an event-frame fusion framework that exploits the complementary nature of two modalities: event cameras capture rapid motion dynamics asynchronously without data redundancy, while conventional low-frame-rate images provide low-noise, spatially dense observations.

The core methodology consists of two main innovations:

  1. Affine Invariant Simplicial (AIS) Framework: The non-linear deformation field of an object's surface is systematically divided into multiple local triangular sub-regions (simplices). Within each triangle, the deformation parameters are linearized using an affine transformation. This reduces a high-dimensional problem into a low-parametric vertex tracking task. Events are then associated with these sub-regions using a "same-side test" and warped to a reference timestamp via barycentric interpolation to maximize event contrast.
  2. Neighborhood-Greedy Optimization Strategy: Long-term dense tracking often suffers from local error accumulation, which can cause global tracking failure. The authors introduce a strategy that evaluates the convergence quality of each sub-region. Well-converged regions are fixed and act as anchors, leveraging physical strain continuity priors to guide and correct their poorly-converged neighbors.

The optimization follows a hierarchical coarse-to-fine paradigm, solving for global rigid self-motion first before iteratively splitting and refining the mesh for fine-grained deformations.

Overview of the key steps in the proposed AIS framework, including event warping, anchor trajectory tracking, and dense sampling.

Datasets

Because no standard event-based datasets existed for VDM tasks, the authors built a custom hardware setup consisting of a Prophesee EVK4 event camera and a 210 fps grayscale camera collocated behind a 50:50 beam splitter.

Using this system, they introduced a new benchmark dataset featuring:

  • Over 120 real-world sequences spanning diverse deformation modes (squeezing, stretching, bending, and cracking).
  • Displacements ranging from small-scale (<20 pixels) to large-scale (100+ pixels).
  • Highly accurate ground-truth deformation fields derived by running state-of-the-art VDM algorithms directly on the high-speed 210 fps video recordings.
The hardware configuration and data transmission topology used to collect the new hybrid event-frame VDM benchmark.

Key Insights and Ideas

  • Local Linearization of Complex Fields: Complex, non-linear global deformations can be effectively modeled by a collection of locally linear, affine-invariant triangular patches. This low-parameter representation successfully mitigates the motion ambiguities typically caused by sparse and noisy event data.
  • Greedy Collaboration Prevents Drift: Instead of optimizing the entire high-dimensional field simultaneously—which is slow and prone to local minima—allowing successful regions to "pull" failing neighbors along via continuity constraints prevents tracking drift. This neighborhood-greedy approach yields a threefold speedup in mean convergence time (from 26.5s down to 7.2s).
  • Efficiency Without Sacrifice: High-frequency deformation dynamics do not strictly require high-speed video. Low-frame-rate frames (e.g., 5 fps) combined with asynchronous event streams retain enough spatiotemporal information to match or exceed high-speed video tracking accuracy.
The neighborhood-greedy optimization strategy compared against vanilla optimization, showing how well-converged regions guide neighbors to prevent error accumulation.

Key Results

The framework was evaluated against several baselines, including OpenCorr (model-based VDM), StrainNet (learning-based VDM), E-RAFT (event-based optical flow), and CoTrackerV3 (long-term point tracking).

  • Tracking Robustness: In scenarios with large continuous displacements (100+ pixels), the proposed method achieved a 65.7% survival rate (the fraction of sequence duration before tracking fails), outperforming the top baseline, CoTrackerV3, by 1.6×.
  • Resource Efficiency: Remarkably, when utilizing a 5 fps frame input alongside the event stream, the method achieved tracking accuracy comparable to traditional methods running on 100 fps video while consuming only 18.9% of the data storage and processing resources.
Qualitative results across diverse scenarios, demonstrating the system's capability to map deformation fields on stretching rubber, cracking wafers, and rolling tires.

Limitations and Future Work

The authors note a few key limitations of the current system:

  • Topological Changes: Because the framework relies heavily on spatial continuity assumptions, it can generate "pseudo-strains" if an object undergoes a topological break, such as a material cracking completely apart.
  • Specularities: Performance degrades in scenes with strong specular reflections, which violate the brightness constancy assumption.

Future Directions: The authors plan to extend the framework to 3D deformation fields using synchronized event camera arrays for volumetric strain analysis. They also aim to explore purely event-driven VDM systems to further minimize data throughput and deploy the framework into real-time closed-loop applications like robotic manipulation of deformable objects.

Summary updated

Abstract

Visual Deformation Measurement (VDM) aims to recover dense deformation fields by tracking surface motion from camera observations. Traditional image-based methods rely on minimal inter-frame motion to constrain the correspondence search space, which limits their applicability to highly dynamic scenes or necessitates high-speed cameras at the cost of prohibitive storage and computational overhead. We propose an event-frame fusion framework that exploits events for temporally dense motion cues and frames for spatially dense precise estimation. Revisiting the solid elastic modeling prior, we propose an Affine Invariant Simplicial (AIS) framework. It partitions the deformation field into linearized sub-regions with low-parametric representation, effectively mitigating motion ambiguities arising from sparse and noisy events. To speed up parameter searching and reduce error accumulation, a neighborhood-greedy optimization strategy is introduced, enabling well-converged sub-regions to guide their poorly-converged neighbors, effectively suppress local error accumulation in long-term dense tracking. To evaluate the proposed method, a benchmark dataset with temporally aligned event streams and frames is established, encompassing over 120 sequences spanning diverse deformation scenarios. Experimental results show that our method outperforms the state-of-the-art baseline by 1.6% in survival rate. Remarkably, it achieves this using only 18.9% of the data storage and processing resources of high-speed video methods.
BibTeX
@misc{wu2026event,
  author = {Yuliang Wu and Wei Zhai and Yuxin Cui and Tiesong Zhao and Yang Cao and Zheng-Jun Zha},
  title = {{Event-based Visual Deformation Measurement}},
  year = {2026},
  eprint = {2602.14376},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.14376},
}
Figure