Abstract
In this paper, we investigate the stability of a class of analytic neural networks with synaptic feedback via event-triggered rules. This model is general and includes the Hopfield neural network as a special case. These event-trigger rules can efficiently reduce the loads of computation and information transmission at the synapses of the neurons. The synaptic feedback of each neuron keeps a constant value based on the outputs of the other neurons at its latest triggering time but changes at its next triggering time, which is determined by a certain criterion. It is proved that every trajectory of the analytic neural network converges to a certain equilibrium under this event-triggered rule for all initial values except a set of zero measure. The main technique of the proof is the Łojasiewicz inequality to prove the finiteness of trajectory length. The realization of this event-triggered rule is verified by the exclusion of Zeno behaviors. Numerical examples are provided to illustrate the efficiency of the theoretical results.
Introduction
In traditional recurrent neural network (RNN) models, such as the classic Hopfield network, neurons are typically assumed to receive continuous, real-time feedback from all other neurons. While mathematically convenient, this assumption is computationally expensive in practical applications. Whether in specialized neuromorphic hardware or distributed simulations, constant communication consumes significant power, bandwidth, and computational cycles.
The challenge lies in transitioning from continuous updates to event-triggered updates: a neuron should only broadcast its state or listen to others when its internal state has changed significantly. However, introducing these communication gaps can compromise mathematical guarantees of stability. Furthermore, poorly designed trigger rules can lead to Zeno behavior, where the system attempts to update itself infinitely fast in a finite time interval, effectively crashing the implementation.
Methodology
The research focuses on a general class of analytic neural networks, introducing a novel asynchronous triggering rule designed to maintain stability while reducing communication overhead.
The Trigger Rule and Stability
Each neuron monitors its own "measurement error"—the difference between its current state and the state it last broadcasted. It only sends an update when this error exceeds a time-varying threshold. To prove stability in this sparse-update environment, the authors employ the Łojasiewicz inequality. Unlike standard Lyapunov methods, which can struggle when a system has a manifold of equilibrium points, the Łojasiewicz inequality allows the authors to prove that the trajectory of the network is of finite length, ensuring it eventually settles into a stable state.
Discrete-Time Monitoring
Recognizing that continuously checking a trigger condition is itself computationally expensive, the authors propose a discrete-time monitoring scheme. This algorithm uses the system's own dynamics to predict the earliest possible time a trigger could occur, allowing the hardware to "sleep" in between checks.
This predictive approach ensures that the system remains efficient without missing critical update windows. By calculating a lower bound for the next event, the system avoids the "digital stutter" of Zeno behavior.
Experimental Validation
The theory was validated using two synthetic numerical benchmarks:
- A 5-Dimensional Analytic Network: Demonstrated the convergence of state components and the reduction in triggering events. Increasing the threshold parameter ($\gamma$) from 0.1 to 0.5 reduced the average number of triggering events from 82.9 to 28.7—a nearly 3x improvement in efficiency.
- A 3-Dimensional Network for Optimization: Showed that the event-triggered system successfully finds local minima of a quadratic objective function, proving the approach is viable for tasks like solving the Traveling Salesman Problem.
Key Insights and Conclusion
The study provides several critical takeaways for the design of efficient neural systems:
- Stability is Robust to Sparsity: Global stability can be maintained without a constant stream of information, provided updates occur when the measurement error reaches a specific threshold.
- Asynchrony as a Feature: The rules are "neuron-wise," allowing each neuron to decide when to trigger independently. This makes the approach highly scalable for decentralized or distributed systems.
- The Zeno Guarantee: The authors proved a positive lower bound for inter-event times, ensuring the algorithm is physically realizable on real hardware.
While the proof of "almost sure stability" covers nearly all initial conditions, future work aims to explore self-triggered control. In such a system, neurons would not monitor their error at all but would instead pre-calculate exactly when their next update should occur, further driving down the energy requirements of complex neural simulations.
Summary updated