Centralized and Decentralized Global Outer-synchronization of Asymmetric Recurrent Time-varying Neural Network by Data-sampling

Wenlian Lu, Ren Zheng, Tianping Chen

Abstract

In this paper, we discuss the outer-synchronization of the asymmetrically connected recurrent time-varying neural networks. By both centralized and decentralized discretization data sampling principles, we derive several sufficient conditions based on diverse vector norms that guarantee that any two trajectories from different initial values of the identical neural network system converge together. The lower bounds of the common time intervals between data samples in centralized and decentralized principles are proved to be positive, which guarantees exclusion of Zeno behavior. A numerical example is provided to illustrate the efficiency of the theoretical results.

Introduction

In the study of Recurrent Neural Networks (RNNs), synchronization—ensuring two different systems or trajectories converge to the same state—is a fundamental goal for applications such as secure communication and parallel computing. While continuous-time models are well-understood, real-world implementations rely on digital, discrete-time iterations.

Traditional approaches typically utilize a constant time-step for discretization. However, this method is often computationally expensive and inefficient, particularly for networks characterized by time-varying parameters or asymmetric connections. This research addresses the challenge of designing adaptive step sizes that trigger updates only when necessary to maintain synchronization, thereby reducing communication overhead.

Methodology

The paper proposes two primary data-sampling principles for asymmetric recurrent time-varying neural networks to move beyond costly fixed-step updates:

  1. Structure-dependent Sampling: Update intervals are determined by the inherent parameters of the neural network, including connection weights and activation function bounds.
  2. State-dependent Sampling: Updates are triggered dynamically based on the "measurement error," defined as the difference between a neuron's current state and its state at the most recent sampling point.

These principles are applied across two distinct architectures:

  • Centralized: A global sequence of time points where all neurons update simultaneously.
  • Decentralized (Push-based): Each neuron maintains its own independent update schedule. When a local triggering condition is met, the neuron updates and "pushes" its state to its neighbors.

To provide flexibility across different network constraints, the researchers derive sufficient conditions for synchronization using three different vector norms: $l_1, l_2,$ and $l_\infty$.

Key Insights and Theoretical Results

Mathematical Guarantees

The core of the research consists of several theorems providing the exact inequalities required to guarantee that any two trajectories from different initial values will converge. Numerical simulations demonstrate that the error between these trajectories decreases exponentially over time across all four proposed sampling rules.

Exclusion of Zeno Behavior

A critical contribution of this work is the proof that the time interval between any two samples has a positive lower bound. In control theory, "Zeno behavior" occurs when a system triggers infinite events in a finite amount of time, making it impossible to implement in physical hardware. By proving its exclusion, the authors ensure that these adaptive algorithms are physically realizable.

Efficiency and Decentralization

The study reveals that adaptive sampling allows the network to maintain synchronization with significantly fewer updates than a fixed-step system. While decentralized systems are shown to be feasible without a common clock, they naturally require more total update events across the network than centralized systems to maintain the same level of global stability. Additionally, simulations indicate that state-dependent rules generally trigger fewer updates than structure-dependent rules.

Limitations and Future Work

The current study focuses on "outer-synchronization" between two identical neural network systems within a deterministic framework. While the mathematical proofs scale theoretically, the practical overhead of calculating triggering conditions in massive-scale networks remains an engineering challenge. Future research may explore "generalized synchronization" between systems with different structures and the extension of these results to systems with stochastic disturbances.

Summary updated

Abstract

In this paper, we discuss the outer-synchronization of the asymmetrically connected recurrent time-varying neural networks. By both centralized and decentralized discretization data sampling principles, we derive several sufficient conditions based on diverse vector norms that guarantee that any two trajectories from different initial values of the identical neural network system converge together. The lower bounds of the common time intervals between data samples in centralized and decentralized principles are proved to be positive, which guarantees exclusion of Zeno behavior. A numerical example is provided to illustrate the efficiency of the theoretical results.
BibTeX
@inproceedings{lu2016centralized,
  author = {Wenlian Lu and Ren Zheng and Tianping Chen},
  title = {{Centralized and Decentralized Global Outer-synchronization of Asymmetric Recurrent Time-varying Neural Network by Data-sampling}},
  booktitle = {Neural Networks},
  year = {2016},
  url = {https://openreview.net/forum?id=7WChQjDV9aO},
}
Figure