Finite-Sample System Identification: An Overview and a New Correlation Method

Algo Carè, Balázs Csanád Csáji, Marco C. Campi, Erik WeyerIEEE Control. Syst. Lett 2018 · DOI · OpenReview

Abstract

System identification traditionally relies on asymptotic theory, which assumes infinite data to provide parameter confidence intervals. In real-world engineering, data is often limited, rendering these approximations unreliable. This paper introduces Sign-Perturbed Correlation Regions (SPCR), a novel framework for constructing exact confidence regions for model parameters using finite samples. By combining correlation-based metrics with norm-based evaluation, SPCR overcomes the conservatism of previous methods like LSCR and the computational complexity of SPS in feedback systems. Experimental results on a simulated bilinear system demonstrate that SPCR provides guaranteed coverage and consistent convergence as sample sizes increase.

Introduction

In the field of system identification, the goal is to estimate the hidden parameters of a dynamical system—such as a robotic arm or a chemical reactor—from noisy input/output data. Traditionally, researchers have relied on asymptotic theory, which posits that with infinite data, parameter estimates follow a predictable Gaussian distribution. However, in practical engineering scenarios, data is finite. For small or medium-sized datasets, asymptotic confidence regions can be misleading, often underestimating the true error or assuming an incorrect shape.

While finite-sample methods like Leave-out Sign-dominant Correlation Regions (LSCR) and Sign-Perturbed Sums (SPS) were developed to address these issues, they possess inherent drawbacks. LSCR often produces overly conservative (excessively large) regions, while SPS can become computationally prohibitive for systems where current outputs depend on past noise, such as ARX models.

Methodology: Sign-Perturbed Correlation Regions (SPCR)

The authors propose a unifying framework based on sign-perturbation. The fundamental principle involves taking the noise sequence of a system and "perturbing" it by randomly flipping the signs of the data points. If a candidate parameter matches the true system parameter, the original data sequence should be statistically indistinguishable from these sign-flipped versions.

The core contribution, Sign-Perturbed Correlation Regions (SPCR), integrates two key strategies:

  1. Correlation Analysis: Derived from LSCR, this evaluates the correlation between noise and past inputs/outputs.
  2. Norm-based Evaluation: Borrowed from SPS, this approach stacks multiple correlations into a vector and applies a norm to evaluate them simultaneously, reducing conservatism.

Significantly, SPCR eliminates the need to generate "alternative output sequences," a complex requirement of previous methods for maintaining statistical validity in feedback systems. This simplification makes the algorithm more accessible and computationally efficient for a broader range of engineering models.

Experimental Validation

The SPCR method was validated using a simulated bilinear system defined by: $Y_t = a^*Y_{t-1} + b^*U_t + \frac{1}{2}U_tN_t + N_t$

This model incorporates both linear dynamics and nonlinear noise interactions. The algorithm was tested with sample sizes of $n = 50, 200,$ and $400$ to observe the behavior of the confidence regions.

Confidence regions generated by SPCR for varying sample sizes (n = 50, 200, 400) around the true parameter value.

As shown in the figure above, the SPCR regions are well-behaved and centered on the true parameters $(a^*, b^*)$. A critical advantage of this method is its "exactness": it provides a mathematically guaranteed probability that the true parameter is contained within the region, regardless of sample size. Furthermore, as the number of observations increases, the regions visibly shrink toward the true parameter value, demonstrating empirical consistency.

Discussion and Conclusion

SPCR represents a significant advancement in finite-sample system identification by providing exact coverage guarantees without the heavy computational burden or conservatism of prior techniques. The method requires minimal assumptions, primarily that the noise is jointly symmetric about zero, avoiding the need for Gaussian or i.i.d. assumptions.

Future research directions include developing formal mathematical proofs for the consistency of SPCR and exploring optimal correlation functions for complex nonlinear systems. While more efficient than its predecessors, the method's reliance on evaluating functions over parameter grids suggests that further optimization for high-dimensional models remains a valuable area of study.

Summary updated

Abstract

Finite-sample system identification algorithms can be used to build guaranteed confidence regions for unknown model parameters under mild statistical assumptions. It has been shown that in many circumstances these rigorously built regions are comparable in size and shape to those that could be built by resorting to the asymptotic theory. The latter sets are, however, not guaranteed for finite samples and can sometimes lead to misleading results. The general principles behind finite-sample methods make them virtually applicable to a large variety of even nonlinear systems. While these principles are simple enough, a rigorous treatment of the attendant technical issues makes the corresponding theory complex and not easy to access. This is believed to be one of the reasons why these methods have not yet received widespread acceptance by the identification community and this letter is meant to provide an easy access point to finite-sample system identification by presenting the fundamental ideas underlying these methods in a simplified manner. We then review three (classes of) methods that have been proposed so far-1) Leave-out Sign-dominant Correlation Regions (LSCR); 2) Sign-Perturbed Sums (SPS); 3) Perturbed Dataset Methods (PDMs). By identifying some difficulties inherent in these methods, we also propose in this letter a new sign-perturbation method based on correlation which overcome some of these difficulties.
BibTeX
@article{care2018finite,
  author = {Algo Carè and Balázs Csanád Csáji and Marco C. Campi and Erik Weyer},
  title = {{Finite-Sample System Identification: An Overview and a New Correlation Method}},
  journal = {IEEE Control. Syst. Lett.},
  year = {2018},
  doi = {10.1109/lcsys.2017.2720969},
}
Figure