Numerical Analysis of Modeling Based on Improved Elman Neural Network

Jie Shao, Wang Li, Zhao WeiSong -, Zhong YaQin, Reza MalekianThe Scientific World JOURNAL 2014 · DOI

TL;DR

This paper introduces an "Improved Elman Neural Network" (IENN) for modeling nonlinear circuits with memory effects, like Class-D power amplifiers. By using Chebyshev orthogonal polynomials as activation functions and adding a self-feedback mechanism to context nodes, IENN achieves higher accuracy and faster convergence than existing models, effectively capturing complex nonlinear behaviors.

What Problem Does This Paper Address?

Modern communication and television systems rely heavily on efficient power amplifiers, such as Class-D power amplifiers (CDPAs). These amplifiers, while power-efficient, exhibit significant nonlinearities and, crucially, memory effects. Memory effects arise because the amplifier's output depends not only on the current input but also on its past states, due to dynamic parameter distributions. This phenomenon can lead to imbalances in distortion products, making accurate modeling challenging. Traditional behavioral modeling techniques, like Volterra series, become computationally complex with many parameters, limiting their applicability to "weakly" nonlinear systems. While neural networks offer a promising avenue for nonlinear function approximation, existing models may not fully capture the intricate memory dynamics of these amplifiers. This paper aims to address this gap by proposing a novel neural network architecture that can accurately model these nonlinearities and memory effects.

Methods

The core contribution of this paper is the development of an "Improved Elman Neural Network" (IENN). The IENN builds upon the basic Elman Neural Network (BENN) architecture, which includes input, hidden, context, and output layers, with a feedback loop from the context layer to the hidden layer to capture temporal dependencies. The key innovations in IENN are:

  • Chebyshev Orthogonal Basis Functions: Instead of the traditional sigmoid activation functions in the hidden layer, IENN utilizes the first category of Chebyshev orthogonal basis functions. These functions are computationally simpler to calculate via recursion and have been shown to improve learning speed and accuracy in other neural network applications.
  • Enhanced Context Layer Feedback: A self-feedback mechanism with a feedback coefficient gain is added to the context layer neurons. This modification aims to increase the "memory depth" of the network, making its output more sensitive to historical inputs and thus better representing memory effects.

The training of the IENN model is performed using a gradient descent (GD) algorithm to minimize the sum of squared error (SSE) between the network's output and the actual system output. The paper also details a systematic approach to determine the optimal number of hidden layer neurons by analyzing SSE curves.

Datasets

The paper does not introduce new datasets. Instead, it uses simulation data generated from a half-bridge Class-D power amplifier (CDPA) circuit. The CDPA circuit is driven by various input signals to generate training and testing data:

  • Two-tone signal: Used for initial analysis of optimal hidden layer neurons and for comparing model performance.
  • Linear Frequency Modulated (LFM) signal: Used for experimental validation.
  • Binary Phase Shift Keying (2PSK) signal: Used for further experimental validation.

The simulation data is generated with specific parameters for each signal type, including frequencies, amplitudes, bandwidths, and sampling frequencies, as detailed in the simulation sections.

Key Insights and Ideas

  • Chebyshev Functions for Efficiency and Accuracy: Replacing sigmoid functions with Chebyshev orthogonal polynomials in the hidden layer not only simplifies computation but also demonstrably improves the accuracy and convergence speed of the neural network model.
  • Enhanced Memory Representation: The added self-feedback in the context layer of the IENN is crucial for effectively capturing and representing the memory effects inherent in nonlinear circuits like CDPA. This allows the model to be more sensitive to past inputs.
  • IENN Outperforms Existing Models: The IENN consistently shows superior performance compared to established models like Volterra-Laguerre (VL), Chebyshev Neural Network (CNN), and the basic Elman Neural Network (BENN) across various input signals and evaluation metrics (time-domain error, frequency-domain spectrum error).
  • Reduced Complexity: IENN requires fewer hidden layer neurons and less training time to achieve comparable or better accuracy than BENN, suggesting a more efficient and simpler network structure.

Key Results

The paper presents simulation results across three different input signal types (two-tone, LFM, and 2PSK) to evaluate the performance of IENN against VL, CNN, and BENN models.

  • Time Domain Accuracy: In time-domain analysis, IENN consistently achieved significantly lower mean and maximum transient errors compared to VL, CNN, and BENN. For instance, with a two-tone signal, the maximum transient error for IENN was on the order of 10⁻⁵ V, while for BENN it was around 0.0391 V. VL and CNN models showed very large errors, indicating they failed to accurately reconstruct the CDPA output.
  • Frequency Domain Accuracy: In frequency-domain analysis, IENN also demonstrated superior performance. The spectrum error for IENN was significantly lower than for VL and CNN, which struggled to represent the intermodulation distortion (IMD) components indicative of nonlinearity and memory effects. While BENN also showed stable spectrum errors, IENN was found to be more accurate, with spectrum errors on the order of 10⁻⁶ dB compared to BENN's 0.011 dB.
  • Convergence and Neuron Count: The study on optimal hidden layer neurons showed that IENN converges faster and requires fewer neurons than BENN to achieve a target error threshold. For example, to reach a specific SSE, IENN needed approximately 25 neurons while BENN required 30.

Limitations and Future Work

  • Limited to Specific Circuit Type: The proposed IENN model is demonstrated for the half-bridge Class-D power amplifier. While the principles are generalizable, further validation on other types of nonlinear circuits with memory effects would be beneficial.
  • Parameter Tuning: The paper mentions that initial values for learning rates (α, η₁, η₂, η₃) and other parameters are obtained through continuous testing. A more systematic or automated approach to parameter tuning could further enhance the model's robustness.
  • Future Work: The authors suggest that the IENN model is effective, efficient, and simple for nonlinear systems. Future work could explore its application to other complex systems, investigate more advanced training algorithms, or explore variations of the IENN architecture for even greater performance.

Summary updated

Abstract

A modeling based on the improved Elman neural network (IENN) is proposed to analyze the nonlinear circuits with the memory effect. The hidden layer neurons are activated by a group of Chebyshev orthogonal basis functions instead of sigmoid functions in this model. The error curves of the sum of squared error (SSE) varying with the number of hidden neurons and the iteration step are studied to determine the number of the hidden layer neurons. Simulation results of the half-bridge class-D power amplifier (CDPA) with two-tone signal and broadband signals as input have shown that the proposed behavioral modeling can reconstruct the system of CDPAs accurately and depict the memory effect of CDPAs well. Compared with Volterra-Laguerre (VL) model, Chebyshev neural network (CNN) model, and basic Elman neural network (BENN) model, the proposed model has better performance.
BibTeX
@article{shao2014numerical,
  author = {Jie Shao and Wang Li and ZHAO WeiSong - and Zhong YaQin and Reza Malekian},
  title = {{Numerical Analysis of Modeling Based on Improved Elman Neural Network}},
  journal = {The Scientific World JOURNAL},
  year = {2014},
  doi = {10.1155/2014/271593},
}
Figure