A new paradigm for parameter estimation in system modeling

Simone Garatti, S. BittantiInternational Journal of Adaptive Control and Signal Processing 2012 · DOI

TL;DR

This paper introduces a novel "two-stage" (TS) parameter estimation paradigm that prioritizes computational efficiency and automation. By training an estimator offline using extensive simulations, the TS approach delivers explicit, ready-to-use estimators that require minimal computational effort during real-time application, addressing a key limitation of traditional methods.

What Problem Does This Paper Address?

System identification, the process of building mathematical models from data, often involves estimating unknown parameters of a system. While numerous methods exist (like Kalman filters, maximum likelihood, and prediction error), they often struggle with computational complexity, requiring significant processing power and sometimes human intervention for tuning. This makes them impractical for real-time applications or embedded systems. The paper highlights that traditional methods, while well-studied for accuracy, often overlook crucial aspects like computational cost and automation, leading practitioners to rely on less rigorous empirical methods. The authors aim to bridge this gap by proposing a method that is both accurate and computationally efficient for practical deployment.

Methods

The core contribution is the Two-Stage (TS) estimation paradigm. This approach fundamentally differs from traditional methods by not defining an estimator through an explicit optimization criterion. Instead, it's an empirical process involving two stages:

  1. Compression Stage (g): The unknown parameter vector $\theta$ is used to generate a large dataset of simulated input/output sequences from the system model. These sequences are then "compressed" into shorter sequences (artificial data) by fitting a simple, intermediate model (e.g., an ARX model) to each simulated data segment. This stage effectively captures the essential information about $\theta$ from the long input/output sequences into a lower-dimensional representation.
  2. Fitting Stage (h): A mapping function (e.g., a neural network) is trained to directly relate these compressed artificial data sequences to the corresponding parameter values $\theta$. This stage identifies the relationship between the compressed data and the parameters.

The final estimator is the composition of these two functions: $f = h \circ g$. This means the estimator is explicitly defined and can be applied directly to new, real-world data without further computation beyond evaluating the trained functions. The computational burden is entirely shifted to the offline training phase.

Datasets

The paper uses two main types of data:

  • Simulated Data: For training the TS estimator, a large number of input/output sequences are generated by simulating the system model with various parameter values $\theta$ sampled from a defined range. In the benchmark example, 1500 such sequences were generated.
  • Benchmark Example Data: A synthetic dynamical system (Equations 5a-5c) is used to generate 200 input/output sequences for each of 200 different true parameter values $\theta$ sampled uniformly from [-0.9, 0.9]. This dataset is used to evaluate the performance of the TS approach against traditional methods.
  • Pacejka Tire Model Data: For the real-world application, simulated data from a car model (using Pacejka's magic formula for tire dynamics) is generated. This involves 2000 simulations for training and 800 validation sequences.

Key Insights and Ideas

  • Shifting the Computational Burden: The most significant insight is moving the computational cost from online (real-time estimation) to offline (training). This is achieved by explicitly constructing the estimator through simulation, making real-time applications extremely fast.
  • "White-Box" Identification with "Black-Box" Flexibility: The TS approach leverages the availability of a system simulator (white-box) but constructs an estimator that behaves like a black-box model, offering flexibility in choosing the fitting function (stage 2).
  • Decoupling Bias and Variance: The two-stage process helps manage the bias-variance trade-off. The first stage reduces dimensionality, and the second stage fits a function to this reduced data, making the fitting problem more tractable and less prone to overfitting.
  • Modularity and Designer Freedom: The TS paradigm offers significant flexibility. Designers can choose different intermediate models for compression and various function classes for fitting, allowing them to tailor the estimator to specific problem requirements and readily iterate on choices.
  • Addressing Local Minima: Unlike methods like Prediction Error (PE) or Maximum Likelihood (ML) that can get trapped in local minima during optimization, the TS approach's minimization occurs during the training of the fitting function (stage 2), which is less susceptible to the same issues related to the original parameter space.

Key Results

  • Computational Efficiency: The TS estimator achieved extremely low computational cost during real-time application. In the benchmark example, it took only 1.95 seconds to generate 200 estimates, averaging about 0.01 seconds per estimate. This is orders of magnitude faster than Particle Filters (PF) and significantly faster than Extended Kalman Filters (EKF) and Unscented Kalman Filters (UKF) in the benchmark.
  • Accuracy: In the benchmark example, the TS estimator provided accurate estimates, comparable to or better than PF, and significantly better than EKF/UKF, especially when those methods struggled with initial conditions. For the Pacejka tire parameter estimation, the TS estimator provided "fairly accurate estimates," particularly for the more influential parameter $\theta_1$.
  • Training Time: The offline training phase for the TS estimator can be computationally intensive. For the benchmark example, training took 31.45 seconds. For the Pacejka model, it took about 20 minutes. However, this is a one-time cost.

Limitations and Future Work

  • Training Data Requirements: The TS approach requires a substantial amount of simulated data for training, which can be computationally expensive and time-consuming. The quality and quantity of this training data are critical for the estimator's performance.
  • Choice of Intermediate Models and Fitting Functions: The performance of the TS estimator depends on the choices made for the compression model (stage 1) and the fitting function (stage 2). The paper acknowledges that there are no simple recipes for these choices, requiring designer expertise and potentially cross-validation.
  • Formal Theoretical Investigation: The authors explicitly state that they do not provide a formal theoretical investigation of the TS method's properties, suggesting this as a direction for future research.
  • Generalization to Complex Systems: While demonstrated on a nonlinear system and a tire model, further exploration of its applicability to more complex, high-dimensional systems would be valuable.

Summary updated

Abstract

SUMMARY In this paper, we consider a basic problem in system identification, that of estimating the unknown parameters of a given model by using input/output data. Available methods (extended Kalman filtering, unscented Kalman filtering, particle filtering, maximum likelihood, prediction error method, etc.) have been extensively studied in the literature, especially in relation to consistency analysis. Yet, other important aspects, such as computational complexity, have been somewhat overlooked so that, when such methods are used in practical problems, remarkable drawbacks may arise. This is why parameter estimation is often performed using empirical procedures. This paper aims to revisit the issue of setting up an estimator that is able to provide reliable estimates at low computational cost. In contrast to other paradigms, the main idea in the new introduced two‐stage estimation method is to retrieve the estimator through simulation experiments in a training phase. Once training is terminated, the user is provided with an explicitly given estimator that can be used over and over basically with no computational effort. The advantages and drawbacks of the two‐stage approach as well as other traditional paradigms are identified with an illustrative example. A more concrete example of tire parameter estimation is also provided. Copyright © 2012 John Wiley & Sons, Ltd.
BibTeX
@article{garatti2012new,
  author = {Simone Garatti and S. Bittanti},
  title = {{A new paradigm for parameter estimation in system modeling}},
  journal = {International Journal of Adaptive Control and Signal Processing},
  year = {2012},
  doi = {10.1002/acs.2335},
}

Loading related content...

Figure