TL;DR
This paper introduces a novel system identification method, Weighted Null Space Fitting (WNSFss), that combines the statistical efficiency of the Prediction Error Method (PEM) with the numerical robustness of Subspace Identification Methods (SIMs). WNSFss achieves asymptotically efficient estimates for state-space models, addressing a long-standing challenge in the field.
What Problem Does This Paper Address?
System identification aims to build mathematical models of dynamic systems from input-output data. Two dominant approaches are the Prediction Error Method (PEM) and Subspace Identification Methods (SIMs). PEM, when using a quadratic cost function, is equivalent to Maximum Likelihood Estimation (MLE) and achieves the theoretical best possible accuracy (asymptotically efficient). However, PEM can suffer from non-convex optimization problems, risking convergence to suboptimal solutions, especially for complex multi-input multi-output (MIMO) systems. SIMs, on the other hand, are known for their numerical stability and ease of use with MIMO systems, but they have historically struggled to achieve asymptotic efficiency, particularly in the presence of external inputs. This paper bridges this gap by proposing a method that offers the statistical advantages of PEM with the numerical robustness of SIMs, aiming for asymptotically efficient state-space model identification.
Methods
The core of the proposed method, WNSFss, is a multi-step least-squares procedure:
- High-Order ARX (HOARX) Modeling: The process begins by fitting a high-order ARX model to the input-output data using Ordinary Least Squares (OLS). This HOARX model serves as an approximate sufficient statistic for the system's dynamics.
- OLS for State Matrix (A): Instead of focusing on the range space of the observability matrix (common in SIMs), WNSFss leverages the null space. It uses the Cayley-Hamilton theorem to establish a linear relationship between the coefficients of the characteristic polynomial of the state matrix (A) and the null space of the extended observability matrix. This relationship is then used to obtain an initial OLS estimate of the coefficients of A.
- WLS for State Matrix (A): The OLS estimate is refined using Weighted Least Squares (WLS). The weighting matrix is derived from the asymptotic distribution of the estimation errors from the HOARX step, allowing for an asymptotically efficient estimate of the coefficients of A.
- OLS for Input/Output Matrices (B, K): With the state matrix A estimated, the method proceeds to estimate the input and output matrices B and K using OLS, leveraging a linear relationship derived from the extended observability matrix.
- WLS for Input/Output Matrices (B, K): Similar to step 3, the OLS estimates of B and K are refined using WLS, again utilizing an optimal weighting derived from the asymptotic properties of the estimation errors.
The method is extended to MIMO systems by introducing canonical parametrizations and adapting the null-space fitting approach.
Datasets
The paper evaluates the proposed WNSFss method on:
- Numerical Examples: Two specific numerical examples are used: a SISO ARMAX model and a SIMO state-space model.
- Benchmark Problems from DaISy: The authors test WNSFss on eight benchmark problems from the DaISy database, which includes SISO, MISO, and MIMO systems.
- Random Systems: Simulations are performed on randomly generated SISO and MIMO systems to assess robustness.
Key Insights and Ideas
- Bridging PEM and SIMs: The most significant insight is the successful integration of the statistical efficiency of PEM with the numerical robustness of SIMs. WNSFss achieves this by focusing on the null space of the observability matrix, a departure from traditional SIM approaches.
- Null Space Fitting for Efficiency: By working with the null space, WNSFss enables an explicit derivation of optimal weighting matrices, which is crucial for achieving asymptotic efficiency. This contrasts with many SIMs where optimal weighting is an open problem.
- Asymptotic Efficiency for State-Space Models: The paper provides a rigorous statistical analysis demonstrating that WNSFss is not only consistent but also asymptotically efficient for identifying state-space models, a goal that has been elusive for SIMs.
- Multi-Step Least Squares as a Powerful Framework: The multi-step least-squares approach, starting with a HOARX model and progressively refining estimates, proves to be a robust and effective strategy for state-space system identification.
Key Results
- Asymptotic Efficiency: Through theoretical analysis and simulations, WNSFss is shown to be asymptotically efficient for both open-loop and closed-loop data, matching the Cramér-Rao Lower Bound (CRLB).
- Competitive Performance: In numerical simulations and on benchmark datasets, WNSFss demonstrates performance comparable to or better than state-of-the-art methods, including PEM (especially when initialized with true parameters) and various SIMs.
- Robustness: The method shows robustness in identifying high-order systems and performs well across different noise levels and system types (SISO, MISO, MIMO).
- MIMO Extension: The paper successfully extends the WNSFss framework to MIMO systems, addressing the complexities of canonical parametrization.
Limitations and Future Work
- System Order Assumption: The current WNSFss method assumes the system order ($n_x$) is known in advance. The paper acknowledges that determining the system order is an important aspect that requires further investigation.
- Kronecker Index Selection (MIMO): For MIMO systems, the choice of Kronecker index for canonical parametrization can influence performance. While the paper suggests enumerating possibilities, more efficient selection strategies could be explored.
- Computational Cost: While generally more efficient than PEM, the multi-step nature and potential for MIMO parametrization enumeration might lead to higher computational costs compared to simpler SIMs in some cases.
The authors suggest future work on order selection and more efficient Kronecker index selection for MIMO systems.
Summary updated