Statistical efficiency of structured CPD estimation applied to Wiener-Hammerstein modeling

José Henrique de Morais Goulart, Maxime Boizard, Rémy Boyer, Gérard Favier, Pierre Comon

Introduction

This research explores the intersection of multi-way array processing (tensor decomposition) and nonlinear system identification. Published in 2015, the paper provides a theoretical benchmark for identifying a specific class of nonlinear systems and evaluates which algorithms can achieve the fundamental limits of estimation accuracy.

The Wiener-Hammerstein Model

Identifying the parameters of a nonlinear dynamical system is a classic challenge in engineering. A popular model for such systems is the Wiener-Hammerstein (WH) model, which consists of a "sandwich" structure: a linear filter, followed by a memoryless nonlinearity, followed by another linear filter.

Block diagram of a Wiener-Hammerstein system, showing the sequence of linear and nonlinear components.

While WH models are powerful, identifying their parameters from noisy data is difficult. A modern approach involves estimating the system's Volterra kernels (higher-order generalizations of impulse responses) and treating them as tensors. These tensors have a specific structure: they can be decomposed using a Canonical Polyadic Decomposition (CPD) where the factors are "banded circulant" matrices representing the coefficients of the linear filters.

Methodology and the Cramér-Rao Bound

The authors frame the identification as a structured CPD estimation problem. By imposing the "banded circulant" constraint, the number of parameters to estimate is drastically reduced compared to an unstructured tensor decomposition.

To establish a performance benchmark, the authors derive a closed-form Cramér-Rao Bound (CRB) using the Slepian-Bangs formula and the Jacobian of the model. This bound represents the minimum possible variance for any unbiased estimator. They utilize oblique projections to separate the contributions of different parameter sets (the two linear filters) to the total error bound.

Comparative Analysis of Estimators

The study compares several estimation strategies through Monte Carlo simulations:

  • CALS (Circulant-constrained Alternating Least Squares): A standard iterative approach adapted for this structure.
  • CPTOEP: A non-iterative, algebraic method designed for Toeplitz/banded structures.
  • Refined Methods: These use CPTOEP to provide a "good" starting point and then refine it using either CALS or a quasi-Newton method (BFGS).

Results and Statistical Efficiency

The results demonstrate a significant gap between standard iterative methods and the theoretical limit. Standard methods like CALS are highly sensitive to initialization; even with multiple random restarts (10-CALS), the algorithm often fails to find the global optimum.

Comparison of Mean Square Error (MSE) for various estimators against the theoretical Cramér-Rao Bound (CRB).

As shown in the performance table, the CPTOEP-BFGS estimator (algebraic initialization followed by quasi-Newton refinement) is the only method that consistently achieves statistical efficiency. Its MSE matches the CRB across the entire tested Signal-to-Noise Ratio (SNR) range. For instance, at 60 dB SNR, it achieves an MSE of -69.62 dB against a CRB of -70.18 dB, while 10-CALS remains over 5 dB away from the limit.

Conclusion

The primary insight of this work is that while the Wiener-Hammerstein identification problem is well-behaved near the optimum, the "attraction basin" is difficult to hit with random guesses. The combination of an algebraic start (CPTOEP) and local refinement (BFGS) provides a robust and statistically efficient solution for structured tensor-based system identification.

Summary updated

Abstract

The computation of a structured canonical polyadic decomposition (CPD) is useful to address several important modeling problems in real-world applications. In this paper, we consider the identification of a nonlinear system by means of a Wiener-Hammerstein model, assuming a high-order Volterra kernel of that system has been previously estimated. Such a kernel, viewed as a tensor, admits a CPD with banded circulant factors which comprise the model parameters. To estimate them, we formulate specialized estimators based on recently proposed algorithms for the computation of structured CPDs. Then, considering the presence of additive white Gaussian noise, we derive a closed-form expression for the Cramér-Rao bound (CRB) associated with this estimation problem. Finally, we assess the statistical performance of the proposed estimators via Monte Carlo simulations, by comparing their mean-square error with the CRB.
BibTeX
@inproceedings{goulart2015statistical,
  author = {José Henrique de Morais Goulart and Maxime Boizard and Rémy Boyer and Gérard Favier and Pierre Comon},
  title = {{Statistical efficiency of structured CPD estimation applied to Wiener-Hammerstein modeling}},
  booktitle = {EUSIPCO},
  year = {2015},
  url = {https://openreview.net/forum?id=LJKD0a1VXjq},
}
Figure