Robust Pixel-wise Digital Image Correlation under Large Motions and Illumination Variations via a Global-to-Local Hybrid Neural Field

Xianglong Guo, Jianyang LiarXiv 2026 · arXiv

TL;DR

This paper introduces a two-stage hybrid neural framework for Digital Image Correlation (DIC) that overcomes the traditional "Achilles' heels" of the method: large rigid-body motions and fluctuating illumination. By combining a masked global alignment stage with a hybrid Kolmogorov-Arnold Network (KAN) and Multilayer Perceptron (MLP) residual field, the authors achieve robust, pixel-wise displacement measurements even under extreme 150° rotations and significant lighting variations.

What Problem Does This Paper Address?

Digital Image Correlation (DIC) is the workhorse of experimental mechanics, used to measure how materials deform by tracking speckle patterns in images. However, standard DIC faces a persistent trade-off: "local" methods (subset-based) struggle with spatial resolution and noise, while "global" methods (finite-element based) can be computationally heavy and sensitive to initialization.

Recently, "neural DIC" methods like PINN-DIC have emerged, treating displacement as a continuous field learned by a neural network. While promising, these methods often fail when:

  1. Motions are large: If a specimen rotates or translates significantly, the network gets stuck in local minima before it can find the correct alignment.
  2. Lighting changes: Most DIC assumes "brightness constancy," but real-world experiments involve shadows, glare, or intensity shifts that break standard pixel-matching algorithms.

Methods

The authors propose a global-to-local hybrid neural DIC framework that operates in two distinct stages:

Stage 1: Masked Global Rigid Alignment

Before diving into fine-grained deformation, the system performs a "warm-up" to handle large motions. It uses a coarse-to-fine search involving phase correlation to estimate dominant rotation and translation. Crucially, it employs a bidirectional valid-overlap mask. This ensures the optimizer only looks at pixels that actually exist in both the reference and deformed images, preventing errors caused by the "black borders" created when you warp or rotate an image.

Fig. 2. Overview of Stage 1 masked global rigid alignment. A coarse angle search and phase correlation provide the initial

Stage 2: Dense Neural Residual Deformation

Once the images are roughly aligned, a second neural network learns the remaining "residual" (non-rigid) deformation. The architecture is a hybrid KAN-MLP:

  • KAN (Kolmogorov-Arnold Network): A newer alternative to standard MLPs that uses learnable univariate functions on the network edges. Here, it acts as a compact nonlinear feature generator.
  • MLP (Multilayer Perceptron): Refines these features into precise, pixel-wise displacement vectors.
Fig. 3. Overview of Stage 2 dense neural residual deformation estimation. A hybrid KAN-MLP neural field predicts the

The Secret Sauce: Illumination-Robust Structural Loss

Instead of comparing raw pixel values, the authors use a local zero-normalized Sobel structural loss. By focusing on image gradients (edges) rather than absolute brightness, the method becomes largely immune to lighting changes and "haze-like" degradations.

Fig. 4. Illustration of the proposed illumination-robust structural matching loss. The predicted deformed image 퐼￿￿￿ and the

Datasets

The framework was evaluated using:

  • Zhao et al. DIC Benchmark: A standard open-source dataset containing simulated speckle images with known ground-truth displacements (translation, rotation, and tensile deformation).
  • Degraded Image Simulations: Custom-generated images using a haze-like formation model to test robustness against non-uniform intensity changes.
  • Real Experimental Data: A sequence of real speckle images captured under large in-plane rotations (50°, 100°, and 150°) and varying illumination.

Key Insights and Ideas

  • Global-to-Local Strategy is Essential: Neural networks are powerful interpolators but poor "searchers." By separating the search for global motion (Stage 1) from the learning of local deformation (Stage 2), the authors prevent the network from failing on large displacements.
  • Hybrid Architectures Win: The KAN module provides a more flexible, nonlinear basis for representing coordinates than a standard MLP alone, while the MLP head provides the necessary local refinement.
  • Gradients > Pixels: In challenging environments, the structure of the speckle pattern (captured by Sobel gradients) is a much more reliable signal than the intensity of the pixels.

Key Results

  • Accuracy: In noise-contaminated sub-pixel translation tests, the proposed method achieved a Mean Absolute Error (MAE) of 0.0025 pixels, significantly lower than VIC2D (0.0114) and KAN-DIC (0.0329).
  • Robustness to Rotation: While existing neural DIC methods (PINN-DIC, KAN-DIC) failed to converge on a 30° rotation, the proposed method recovered the displacement field with high accuracy, matching or exceeding commercial software (VIC2D).
  • Real-World Performance: In real experiments with a massive 150° rotation, the method maintained excellent gray-field consistency (low GLR), whereas traditional methods began to show significant boundary artifacts.

Limitations and Future Work

  • Computational Speed: Like most optimization-based neural fields, this method is slower than "one-shot" feed-forward deep learning models because it must be optimized for every new image pair.
  • Hyperparameter Tuning: The performance of the KAN-MLP field depends on settings like grid size and network width, which may require manual adjustment for different speckle qualities.
  • Future Directions: The authors plan to improve computational efficiency and incorporate mechanical constraints (like strain-displacement relations) to make the measurements even more physically consistent.

Summary updated

Abstract

Digital image correlation (DIC) is a widely used non-contact technique for full-field displacement measurement. Data-driven deep-learning DIC offers rapid inference but may generalize poorly to unseen speckle patterns, deformation modes, noise levels, and illumination conditions. Classical optimization-based DIC and recent image-constrained neural formulations avoid labeled displacement data, yet their convergence and accuracy remain vulnerable to initialization, large rigid-body motion, fine non-uniform deformation, and gray-value disturbances. This paper presents a global-to-local hybrid neural DIC framework for robust pixelwise measurement. Stage 1 estimates a dominant in-plane rigid transformation through coarse rotation search, phase correlation, and masked refinement, providing a reliable initialization under large translation and rotation. Stage 2 represents the residual displacement with a hybrid Kolmogorov-Arnold network (KAN) and multilayer perceptron (MLP), and optimizes it using a valid-overlap, illumination-robust structural loss. The local gray-value model explicitly accommodates multiplicative and additive intensity changes. Simulated and real experiments demonstrate accurate and stable displacement recovery under noise, non-uniform deformation, large rotation, illumination variation, and degraded imaging conditions.
BibTeX
@misc{guo2026robust,
  author = {Xianglong Guo and Jianyang Li},
  title = {{Robust Pixel-wise Digital Image Correlation under Large Motions and Illumination Variations via a Global-to-Local Hybrid Neural Field}},
  year = {2026},
  eprint = {2611.01018},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.01018},
}
Figure