A Review on Zeroing Neural Networks

Chengze Jiang, Jie Gui, Long Jin, Shuai LiarXiv 2025 · DOI · arXiv

Introduction

Zeroing Neural Networks (ZNNs) represent a specialized class of recurrent neural networks (RNNs) engineered specifically for solving time-varying optimization and control problems in real-time. Unlike traditional optimization algorithms designed for static environments, ZNNs are built to handle dynamic systems where parameters change over time. In scenarios such as robotic arm trajectory tracking or real-time signal processing, conventional methods often suffer from "lagging error"—calculating solutions for past states rather than current ones. ZNNs mitigate this by shifting the objective from minimizing a standard energy function to zeroing a problem-specific error function $e(t)$, incorporating time derivatives to "predict" future states and eliminate theoretical lag.

Methodology

The development of a ZNN model typically follows a unified three-step framework:

  1. Error Function Definition: An error function $e(t)$ is constructed to measure the residual of the target problem (e.g., $Ax - b = 0$).
  2. Evolution Formula Design: A formula $\dot{e}(t) = F(e(t))$ is defined to dictate how the error decays. The choice of $F(\cdot)$ is critical for determining convergence characteristics.
  3. Model Derivation: The error function and evolution formula are combined to derive a model suitable for hardware or software implementation.

Several "flavors" of ZNNs have emerged, each optimizing for different performance metrics, as shown in the evolution functions below:

Evolution functions of various ZNN models
  • OZNN (Original): The baseline model utilizing linear activation.
  • NTZNN (Noise-Tolerant): Incorporates an integration term to suppress external noise.
  • VPZNN (Varying-Parameter): Employs time-varying gains for super-exponential convergence.
  • FTZNN (Finite-Time): Uses non-linear activation functions to ensure error reaches zero within a fixed time limit.

For digital implementation, continuous ZNN models require discretization using methods such as Euler, Taylor-type, or Runge-Kutta.

Applications and Benchmarks

As a comprehensive review, this work surveys the application of ZNNs across diverse mathematical and engineering domains:

  • Matrix Algebra: Solving matrix inversion, QR decomposition, and tensor square roots.
  • Robotics: Managing kinematic control for redundant manipulators and PUMA560 arms.
  • Positioning: Enhancing Angle-of-Arrival (AoA) and Time Delay of Arrival (TDOA) localization.
  • Image Processing: Facilitating denoising and target detection in remote sensing.

Key Insights

A fundamental shift in ZNN design is the move from "correction" to "prediction." By utilizing the time derivative of problem parameters, ZNNs anticipate changes, effectively bypassing the lag inherent in gradient-based networks. Furthermore, the choice of activation function is a primary driver of performance; transitioning from linear to non-convex or sign-power functions can upgrade convergence from "eventual" to "finite-time." Additionally, the introduction of integral feedback in NTZNN architectures has made these networks robust enough for real-world electronic environments where noise is prevalent.

Comparative Results

The theoretical strengths of various ZNN models are summarized in the comparative analysis below, highlighting their convergence properties and robustness.

Comparative analysis of ZNN variants

Most modern variants (NTZNN, VPZNN, FTZNN) achieve global convergence. VPZNN and FTZNN lead in speed, while NTZNN remains the most resilient against constant, linear, and bounded random noise.

Limitations and Future Directions

Despite significant progress, several challenges remain. Advanced acceleration techniques, such as Nesterov’s method, have yet to be fully integrated into the ZNN framework. There is also a trade-off between mathematical complexity and real-time response; non-linear activations increase computational cost per iteration, which may be counterproductive in high-speed systems. Finally, exploring ZNN-based evolution functions as optimizers for deep learning training presents a promising avenue for future research.

Summary updated

Abstract

Zeroing neural networks (ZNNs) have demonstrated outstanding performance on time-varying optimization and control problems. Nonetheless, few studies are committed to illustrating the relationship among different ZNNs and the derivation of them. Therefore, reviewing the advances for a systematical understanding of this field is desirable. This paper provides a survey of ZNNs' progress regarding implementing methods, analysis theory, and practical applications.
BibTeX
@misc{jiang2025review,
  author = {Chengze Jiang and Jie Gui and Long Jin and Shuai Li},
  title = {{A Review on Zeroing Neural Networks}},
  year = {2025},
  eprint = {2507.00387},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.00387},
}
Figure