Abstract
In this paper, we show that the common approach for simulation non-linear stochastic models, commonly used in system identification, via setting the noise contributions to zero results in a biased response. We also demonstrate that to achieve unbiased simulation of finite order NARMAX models, in general, we require infinite order simulation models. The main contributions of the paper are two-fold. Firstly, an alternate representation of polynomial NARMAX models, based on Hermite polynomials, is proposed. The proposed representation provides a convenient way to translate a polynomial NARMAX model to a corresponding simulation model by simply setting certain terms to zero. This translation is exact when the simulation model can be written as an NFIR model. Secondly, a parameterized approximation method is proposed to curtail infinite order simulation models to a finite order. The proposed approximation can be viewed as a trade-off between the conventional approach of setting noise contributions to zero and the approach of incorporating the bias introduced by higher-order moments of the noise distribution. Simulation studies are provided to illustrate the utility of the proposed representation and approximation method.
The Challenge of Non-linear Simulation
In system identification, mathematical models are constructed from observed data. To validate these models, researchers typically run simulations: the model is provided with an input and allowed to generate its own output over time without correction from real-world measurements.
For linear systems, calculating the "deterministic response" is straightforward—one simply sets the noise terms to zero. However, for non-linear models like NARMAX (Non-linear AutoRegressive Moving Average with eXogenous inputs), noise does not merely add variance; it interacts with the non-linearities to shift the entire mean of the output. If noise is ignored during simulation, the results become systematically biased. The fundamental difficulty lies in the fact that calculating the "true" expected response of these models often requires an infinite number of terms (an infinite impulse response), which is computationally infeasible.
Proposed Methodology
To address this bias, a two-step solution is proposed:
- Hermite Polynomial Representation: Instead of using standard polynomials (e.g., $x^2, x^3$), the model is rewritten using Hermite polynomials. These functions are orthogonal with respect to the Gaussian distribution. Crucially, the expected value of any Hermite polynomial of order 1 or higher is exactly zero. By transforming a model into this domain, finding the unbiased simulation response becomes a matter of "switching off" specific terms.
- The $l$-approximate Simulation Model: For models with non-linear output feedback, simulation creates an infinite chain of dependencies. The authors introduce a "recursive substitution" method that expands the model $l$ times and approximates the remaining infinite "tail" using the simulation's own past values. This provides a tunable trade-off between mathematical accuracy and computational efficiency.
Experimental Validation
The proposed methods were tested using synthetic numerical simulations to provide a "ground truth" for comparison against an ensemble average (the average of thousands of noisy system runs).
In Example 1, a non-linear model where noise affects only the input was examined. The results demonstrated that the conventional "zero-noise" approach resulted in an RMS error of 1.0001, while the proposed Hermite method achieved a significantly lower error of 0.0544.
In Example 2, a more complex model with non-linear output feedback ($y^2_{k-1}$) was used to test the recursive approximation. The $l$-approximate model (using $l=1$ with tail approximation) reduced simulation error from 0.1165 (standard approach) to 0.0348. As the approximation parameter $l$ increases, the simulation response converges toward the true ensemble average.
Key Insights and Limitations
This research highlights that in non-linear systems, the statistical moments of noise (such as variance) contribute directly to the system's deterministic trend. Ignoring noise results in a constant scalar offset in simple cases, but in feedback systems, this bias can propagate and grow, leading to failed model validation.
While effective, the current framework relies on the assumption that noise is Gaussian. Extending this to other distributions increases mathematical complexity. Additionally, as the recursion depth ($l$) increases, the number of terms in the simulation model grows, which may present a memory bottleneck for very high-order polynomials. Future work may also need to address how these approximations affect numerical stability over long time horizons.
Summary updated