A Neural Network Training Method Based on Neuron Connection Coefficient Adjustments

Kun Jiang

TL;DR

This paper introduces a biologically inspired neural network framework based on symmetric differential equations that ditches the traditional chain derivative rule (backpropagation) in favor of "Backward Signal Propagation." By treating neurons as dynamic systems rather than static functions, the authors achieve effective learning on the MNIST dataset even with extremely sparse connectivity—a feat that typically causes standard multilayer perceptrons to fail.

What Problem Does This Paper Address?

Modern deep learning is incredibly successful, but it remains a mathematical abstraction far removed from biological reality. Traditional networks rely on the chain rule for backpropagation, which requires global knowledge of the network and continuous derivative operations—things biological neurons don't naturally do.

Furthermore, while there is high interest in biologically plausible models like Spiking Neural Networks (SNNs), they often struggle with efficiency and scalability. The author, Jiang Kun, aims to bridge this gap by creating a "Wuxing" neural network: a system that is mathematically robust, biologically interpretable, and inherently hardware-friendly by using local signal processing instead of global optimization.

Methods

The core of this research is the Wuxing neural network, which replaces standard numerical neurons with systems governed by symmetric differential equations.

Key technical innovations include:

  • Inherent Nonlinearity: Unlike traditional MLPs, these neurons don't need explicit activation functions (like ReLU or Sigmoid). The differential equations themselves are inherently nonlinear and bounded.
  • Reversibility: The system is designed to be reversible. This allows error signals to propagate backward through the same structure used for forward propagation, eliminating the need for complex partial derivatives.
  • Backward Signal Propagation (BSP): Instead of calculating gradients, the network compares forward and backward signals. It calculates a "causal quantity" ($G_1$) based on the integral of these signals over time and uses an inverse tangent function to limit the adjustment step, updating connection coefficients ($C$) via simple addition.
  • Sparse Connectivity: The model uses a "Link Ratio" (LR) to define connectivity. In their primary test model (MODEL1), the LR was a mere 0.51%—meaning each neuron only communicates with a tiny fraction of its neighbors.

Datasets

The researchers validated their approach using the MNIST dataset (handwritten digits). While a standard benchmark, the focus here wasn't just on top-tier accuracy, but on proving that the network could learn under extreme constraints that would break a traditional MLP.

Key Insights and Ideas

  • Ripples in a Pond: The author uses a beautiful analogy: signal propagation in this network is like throwing a stone into still water. The "ripples" (deviations from a fixed equilibrium point) carry the information through the network.
  • The Sparsity Threshold: One of the most striking findings is the analysis of the "Link Ratio." The paper demonstrates that while standard MLPs start to fail when connectivity drops below 10% (and become nearly untrainable below 3%), the Wuxing network remains functional and robust at just 0.51% connectivity.
  • Hardware-First Design: Because each neuron only processes local signals from its immediate neighbors, the architecture is "hardware-oriented." In a physical circuit implementation, the differential equations would evolve autonomously, removing the computational bottleneck of solving them via software simulation.
  • Hybrid Training: The paper shows that adjusting connection coefficients (like an MLP) is fully compatible with the author's previous work on adjusting "fixed points" (equilibrium states). Combining both methods ("Mixed method") yielded the best accuracy.

Key Results

  • Training Success: The BSP method effectively replaced traditional backpropagation. Accuracy curves show steady improvement across 20-40 training iterations.
  • Model Comparison: MODEL2 (with an optimized neuron distribution: {784, 1048, 353, 119, 40, 10}) outperformed MODEL1, despite having an even lower link ratio of 0.38%. This suggests that the strategy of connection matters more than the sheer number of connections.
  • Robustness: The system successfully avoided local minima by using an additive adjustment strategy for coefficients rather than the multiplicative approach common in early versions of the theory.

Limitations and Future Work

The primary limitation identified is the extremely low connectivity. While the model is robust, the 0.51% link ratio eventually plateaus the learning capacity.

To address this, the author proposes:

  1. Synaptic Adders: Incorporating "adders" at the neuron synapses to allow a single neuron to receive signals from multiple sources without violating the underlying differential equations.
  2. Structural Optimization: Further exploring how the distribution of neurons across layers affects learning efficiency.
  3. Constraint-Based Learning: Continuing to develop training methods that work under strict biological and hardware constraints to better reflect the efficiency of natural nervous systems.

Summary updated

Abstract

In previous studies, we introduced a neural network framework based on symmetric differential equations, along with one of its training methods. In this article, we present another training approach for this neural network. This method leverages backward signal propagation and eliminates reliance on the traditional chain derivative rule, offering a high degree of biological interpretability. Unlike the previously introduced method, this approach does not require adjustments to the fixed points of the differential equations. Instead, it focuses solely on modifying the connection coefficients between neurons, closely resembling the training process of traditional multilayer perceptron (MLP) networks. By adopting a suitable adjustment strategy, this method effectively avoids certain potential local minima. To validate this approach, we tested it on the MNIST dataset and achieved promising results. Through further analysis, we identified certain limitations of the current neural network architecture and proposed measures for improvement.
BibTeX
@misc{jiang2025neural,
  author = {Kun Jiang},
  title = {{A Neural Network Training Method Based on Neuron Connection Coefficient Adjustments}},
  year = {2025},
  eprint = {2502.10414},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.10414},
}

Loading related content...

Figure