A minimal dynamical system and analog circuit for non-associative learning

Matthew Smart, Stanislav Y. Shvartsman, Martin Mönnigmann

Abstract

Learning in living organisms is typically associated with networks of neurons. The use of large numbers of adjustable units has also been a crucial factor in the continued success of artificial neural networks. In light of the complexity of both living and artificial neural networks, it is surprising to see that very simple organisms -- even unicellular organisms that do not possess a nervous system -- are capable of certain forms of learning. Since in these cases learning may be implemented with much simpler structures than neural networks, it is natural to ask how simple the building blocks required for basic forms of learning may be. The purpose of this study is to discuss the simplest dynamical systems that model a fundamental form of non-associative learning, habituation, and to elucidate technical implementations of such systems, which may be used to implement non-associative learning in neuromorphic computing and related applications.

Introduction

In modern artificial intelligence, learning is often synonymous with massive neural networks. However, nature demonstrates that even unicellular organisms without a single neuron can "learn" to ignore repetitive, harmless stimuli—a process known as habituation. This research seeks the "atomic unit" of non-associative learning by identifying the simplest possible dynamical system and analog circuit capable of replicating this behavior. By stripping away the complexity of traditional neural architectures, the authors propose a path toward ultra-low-power "physical learning" systems integrated directly into hardware.

The Minimalist Circuit Design

The authors propose a minimalist circuit where a simple RC (resistor-capacitor) subcircuit acts as a "leaky memory," tracking the history of the input signal $u(t)$. A diode $D$ is then used to compare the current input to this stored memory $x(t)$.

Figure 2: (a) The basic habituation circuit. (b) A variant with a voltage divider. (c) A circuit with a reset switch for dishabituation.

The circuit operates by essentially performing a subtraction: the output $y(t)$ is proportional to the difference between the input and the memory. As the capacitor charges from repeated stimuli, the difference between the input and the memory shrinks, leading to a diminished output—the fundamental mechanism of habituation.

Mathematical Foundations and Nonlinearity

A primary theoretical contribution of this work is the proof that nonlinearity is a strict requirement for habituation. The authors demonstrate that a Linear Time-Invariant (LTI) system cannot habituate; standard linear filters (resistors, capacitors, and inductors) alone are insufficient. A nonlinear element, such as the diode shown in the circuit diagrams, is required to "gate" the response. To bridge this with modern machine learning, the authors show that this behavior can be accurately modeled using the ReLU (Rectified Linear Unit) function, the standard activation function in deep learning.

Behavioral Hallmarks and Results

The proposed system was tested against the established hallmarks of habituation (Thompson and Spencer, 1966). The circuit successfully demonstrated:

  • H1 (Habituation): Response decreases over time with repeated stimulation.
  • H2 (Spontaneous Recovery): If the stimulus stops, the internal state decays, and the response returns to full strength.
Figure 3: Simulation results showing input u(t), internal state x(t), and output y(t) demonstrating habituation and recovery.

The simulations also revealed a phenomenon called "subliminal accumulation." Even after the output has dropped to zero (full habituation), the internal memory continues to build if the stimulus persists. This explains why over-stimulated biological systems often take longer to recover, as the internal state must decay from a higher level before the system becomes sensitive again.

Discussion and Future Outlook

By proving that a single diode and RC pair can perform fundamental learning, this study suggests that learning capabilities can be built into the very fabric of sensors and materials without a central CPU. While the current model is a Single-Input-Single-Output (SISO) system, it serves as a foundational building block. Future work involves connecting these blocks in series or parallel to create complex, non-neuronal learning systems capable of sophisticated pattern recognition and unsupervised clustering with extreme energy efficiency.

Summary updated

Abstract

Learning in living organisms is typically associated with networks of neurons. The use of large numbers of adjustable units has also been a crucial factor in the continued success of artificial neural networks. In light of the complexity of both living and artificial neural networks, it is surprising to see that very simple organisms -- even unicellular organisms that do not possess a nervous system -- are capable of certain forms of learning. Since in these cases learning may be implemented with much simpler structures than neural networks, it is natural to ask how simple the building blocks required for basic forms of learning may be. The purpose of this study is to discuss the simplest dynamical systems that model a fundamental form of non-associative learning, habituation, and to elucidate technical implementations of such systems, which may be used to implement non-associative learning in neuromorphic computing and related applications.
BibTeX
@misc{smart2024minimal,
  author = {Matthew Smart and Stanislav Y. Shvartsman and Martin Mönnigmann},
  title = {{A minimal dynamical system and analog circuit for non-associative learning}},
  year = {2024},
  eprint = {2405.05770},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2405.05770},
}
Figure