Abstract
By prefiltering the input/output data and employing certain decentralized estimation techniques, it is possible to improve the robustness of some estimators significantly. Earlier papers on these techniques have been focused on local convergence properties of certain bootstrap estimators based upon these techniques. This paper is devoted to (1) global convergence properties, and (2) convergence rates when the underlying system is stiff.
Introduction
In the field of system identification, standard techniques such as Least Squares (LS) or Instrumental Variable (IV) methods are the workhorses for estimating model parameters from input-output data. However, these methods often encounter significant numerical instability and slow convergence when applied to "stiff" systems. A stiff system is characterized by dynamics operating on widely disparate time scales—mathematically represented by poles grouped into distinct sets (e.g., some very close to the unit circle and others closer to the origin in the discrete-time domain).
This paper investigates a class of robust estimators that utilize data prefiltering and decentralized estimation to overcome these challenges. Building on previous work that established local stability, this research provides a rigorous analysis of global convergence and examines how system stiffness affects the rate of convergence.
Methodology: Prefiltering and Decentralized Estimation
The core strategy involves polynomial factorization. Rather than attempting to estimate the entire system transfer function in a single step, the system polynomials $A(q^{-1})$ and $B(q^{-1})$ are factored into components (e.g., $A = A_1 \cdot A_2$).
The estimation process follows a bootstrap approach:
- Prefiltering: Input and output data are passed through filters derived from current parameter estimates. For instance, a filtered variable $w_t$ is generated by applying the polynomial $A_2$ to the output $y_t$.
- Decentralized Estimation: This prefiltering transforms the complex, high-order estimation problem into a set of simpler, reduced-order models.
- Recursive Update: Since the "true" filters are unknown, the algorithm iteratively uses current estimates to filter data, which in turn is used to refine the estimates.
To analyze the global convergence of this recursive stochastic algorithm, the authors employ the ODE method (pioneered by Lennart Ljung). This technique associates the discrete-time algorithm with a continuous-time differential equation, allowing the stability of the estimator to be studied through the lens of Lyapunov stability theory.
Global Convergence and the Impact of Stiffness
A primary contribution of this work is the proof of global convergence for the recursive Least Squares variant of the prefiltered estimator.
Theorem 1: Global Convergence
The authors demonstrate that, provided the system is asymptotically stable and the system polynomials are coprime (sharing no common factors), the estimator converges with probability 1 to the true parameter values. This is a significant advancement over previous studies that only guaranteed local convergence—where convergence was only certain if the initial guess was already near the truth.
Theorem 2: Convergence Rates in Stiff Systems
Perhaps the most counterintuitive finding is the relationship between system stiffness and convergence speed. In traditional numerical analysis, stiffness is typically a source of error and instability. However, for these prefiltered estimators, stiffness acts as a benefit. The authors show that as the "stiffness ratio"—the separation between fast and slow dynamics—increases, the eigenvalues of the convergence matrix $F$ tend toward zero.
This leads to a remarkable corollary: for extremely stiff systems, the convergence of the batch variants becomes "instantaneous" in a mathematical sense. By effectively decoupling the dynamics through prefiltering, the algorithm focuses on one set of parameters while remaining "blind" to the interference of the other, thereby avoiding the ill-conditioning that plagues standard LS methods.
Limitations and Considerations
While the results are promising, several constraints remain:
- Coprimeness Requirement: The global convergence proof necessitates that the polynomials are coprime. If common zeros and poles exist, the estimator may converge to non-hyperbolic stationary points, resulting in inaccurate estimates.
- IV Variant Complexity: While the LS variant is proven globally convergent, the Instrumental Variable (IV) variant—though effective in practical simulations (Henriksen 1988, 1989)—lacks a formal global convergence proof in this specific framework.
- A Priori Knowledge: The method requires some initial understanding of the system's dynamic separation to configure the filters correctly. If the dynamics are not sufficiently separated, the advantages of prefiltering diminish, and standard estimators may be more appropriate.
Conclusion
This paper provides a theoretical foundation for the use of prefiltered, decentralized estimators in challenging numerical environments. By proving global convergence and demonstrating that system stiffness actually accelerates the estimation process, the authors offer a robust alternative to standard identification techniques for complex, multi-scale systems.
Summary updated