A training algorithm for networks of high-variability reservoirs

Matthias Freiberger, Peter Bienstman, Joni DambreScientific 2020 · DOI

TL;DR

This paper introduces a novel training algorithm for multi-reservoir computing systems, particularly those with inherent physical variability. By leveraging backpropagation to derive intermediate target signals, their method enables efficient training of complex reservoir networks, potentially overcoming limitations of traditional single-reservoir approaches.

What Problem Does This Paper Address?

Reservoir computing, a brain-inspired approach, offers a promising path to low-energy, high-performance computing by utilizing the complex dynamics of nonlinear systems. While successful with single reservoirs, scaling up performance by simply making them larger hits diminishing returns due to state correlations and hardware constraints. The authors argue for a shift towards multi-reservoir and deep reservoir computing architectures. However, a major hurdle is training these complex systems, especially when the physical reservoirs exhibit inherent variability (e.g., due to fabrication processes) which makes direct application of error backpropagation impossible. Traditional methods like linear regression require known target signals for each reservoir, which are unavailable for intermediate stages in a multi-reservoir setup.

Methods

The core of the paper's contribution is a new training framework that bridges the gap between the power of backpropagation and the practicality of training physical reservoirs. The proposed approach works in simulation:

  1. Architecture Definition: A suitable multi-reservoir architecture is defined for a given task.
  2. Intermediate Target Derivation: Backpropagation is used to train a simulated multi-reservoir network. Crucially, instead of training the entire network to produce the final output, backpropagation is used to derive intermediate target signals for each reservoir in the network.
  3. Training with Intermediate Targets: These derived intermediate targets are then used to train new, potentially physically variable, reservoir networks using classical linear regression.

This method effectively uses backpropagation as a "teacher" to generate training signals for simpler, classical training algorithms, circumventing the direct applicability issues of backpropagation on high-variability systems. For their experiments, they use Echo State Networks (ESNs) as a software surrogate for physical reservoirs.

Datasets

The primary dataset used for training and evaluation is the Nonlinear Autoregressive Moving Average (NARMA-10) system identification task. This task involves predicting the next value in a sequence based on past values and a random input. The authors specifically chose NARMA-10 because it can be analytically decomposed into subtasks, allowing for the creation of "analytical targets" which serve as a strong baseline for comparison.

Key Insights and Ideas

  • Backpropagation as a Target Generator: The most significant insight is using backpropagation not for direct end-to-end training of physical reservoirs, but as a sophisticated method to generate appropriate training signals (intermediate targets) for them. This circumvents the need for direct gradient information from the physical system.
  • Bridging Simulation and Reality: The approach elegantly bridges the gap between simulation (where backpropagation is feasible) and real-world physical systems (which may have inherent variability and limitations).
  • Multi-Reservoir Advantage: The work provides evidence that multi-reservoir architectures, when properly trained, can outperform single, larger reservoirs, even with the same total number of nodes. This suggests a path to increased performance beyond simple scaling.
  • Learned Targets Mimic Analytical Ones: The intermediate targets learned via backpropagation closely resemble the analytically derived targets, validating the approach's ability to capture the underlying task structure.

Key Results

  • Performance Improvement: The multi-ESN system trained with analytically derived targets achieved a Normalized Mean Squared Error (NMSE) of 0.020 ± 0.013, outperforming a monolithic ESN baseline with an NMSE of 0.035 ± 0.016.
  • Learned Targets Effectiveness: When trained with targets derived by their proposed method (Models 1-5), the multi-ESN systems achieved NMSEs ranging from 0.031 ± 0.003 (Model 2) to 0.072 ± 0.017 (Model 1). Notably, the systems trained using these learned targets generally performed better than the models that generated those targets.
  • Model 2's Performance: Model 2, which generated targets that showed strong correlation with the analytical targets, led to the best performance among the learned target models, achieving an NMSE of 0.031 ± 0.003 when used to train new reservoirs.
  • Proof of Concept: The method successfully demonstrated that backpropagation can derive intermediate targets for multi-reservoir networks, which can then be used to train new networks efficiently.

Limitations and Future Work

  • Simulation-Heavy: The current approach relies heavily on simulation for architecture definition and target derivation. Transferring this to actual physical hardware requires careful consideration of hardware-specific challenges.
  • Architecture Search: The paper assumes a given architecture. Future work should explore methods for automatically finding optimal multi-reservoir architectures, similar to deep learning's architecture search.
  • Refining Backpropagation: The authors suggest optimizing the backpropagation process for target derivation, potentially by extending it across multiple layers of reservoirs.
  • Physical Reservoir Transfer: Applying the method to actual physical reservoirs (photonic, spintronic, etc.) will require addressing substrate-specific challenges, such as controlling reservoir dynamics when parameters are not directly tunable.
  • Task Complexity: While effective for NARMA-10, the authors acknowledge that this task is considered "largely solved." The true potential of their method might be more pronounced on harder, unsolved tasks where analytical decomposition is not feasible.

Summary updated

Abstract

Abstract Physical reservoir computing approaches have gained increased attention in recent years due to their potential for low-energy high-performance computing. Despite recent successes, there are bounds to what one can achieve simply by making physical reservoirs larger. Therefore, we argue that a switch from single-reservoir computing to multi-reservoir and even deep physical reservoir computing is desirable. Given that error backpropagation cannot be used directly to train a large class of multi-reservoir systems, we propose an alternative framework that combines the power of backpropagation with the speed and simplicity of classic training algorithms. In this work we report our findings on a conducted experiment to evaluate the general feasibility of our approach. We train a network of 3 Echo State Networks to perform the well-known NARMA-10 task, where we use intermediate targets derived through backpropagation. Our results indicate that our proposed method is well-suited to train multi-reservoir systems in an efficient way.
BibTeX
@inproceedings{freiberger2020training,
  author = {Matthias Freiberger and Peter Bienstman and Joni Dambre},
  title = {{A training algorithm for networks of high-variability reservoirs}},
  booktitle = {Scientific},
  year = {2020},
  doi = {10.1038/s41598-020-71549-y},
}
Figure