Identification of Probability weighted ARX models with arbitrary domains

Alessandro Brusaferri, Matteo Matteucci, Stefano SpinelliarXiv 2020 · arXiv

Abstract

Hybrid systems, which switch between different linear behaviors based on operating conditions, are prevalent in cyber-physical systems. Traditional PieceWise AutoRegressive with eXogenous input (PWARX) models often struggle with nonlinear boundaries between these regimes, requiring redundant sub-models to approximate complex transitions. This paper proposes the Nonlinearly Probabilistic ARX (NPrARX) framework, which utilizes a neural network gating function within a Mixture of Experts architecture to learn arbitrary, nonlinear boundaries directly from data. Using an Expectation-Maximization (EM) algorithm, the framework concurrently identifies system dynamics and switching logic. Experimental results on a complex synthetic benchmark demonstrate high precision in both parameter estimation and mode identification.

Introduction

Many real-world "Cyber-Physical Systems," such as robotic arms hitting physical limits or chemical plants switching valves, exhibit hybrid dynamics. These systems behave linearly for periods before suddenly shifting to a different regime based on specific conditions. The standard tool for modeling such behavior is the PWARX model. However, traditional PWARX identification assumes that the boundaries between behaviors are polyhedral—essentially straight lines or flat planes.

In practice, these boundaries are often nonlinear. Forcing linear partitions onto curved boundaries leads to model over-complexity, as extra sub-models must be invented just to approximate the curves. The NPrARX framework addresses this by allowing the model to learn curved, arbitrary boundaries directly from data, leading to more parsimonious and interpretable models.

Methodology

The NPrARX approach treats system identification as a Mixture of Experts (MoE) task consisting of two primary components:

  1. The Experts: These are standard ARX sub-models (linear equations) that describe the system's dynamics within specific regions.
  2. The Gate: This component determines which expert is active. By using a Feed-Forward Neural Network with tanh activation functions as the gating mechanism, the model can draw complex, curved boundaries in the regressor space.

Training is performed using the Expectation-Maximization (EM) algorithm. In the E-step, the model calculates the posterior probability that each expert was responsible for a specific output. In the M-step, the ARX parameters are updated via weighted least squares, and the neural network weights are optimized using the Adam optimizer to match the "soft labels" generated in the E-step. To avoid local minima where experts model the same average behavior, the authors employ high-variance random initialization for the neural network weights.

Experimental Results

The framework was evaluated using a synthetic benchmark designed to test hybrid identification under non-linearly separable conditions. The system consists of two distinct affine maps arranged such that a standard PWARX model would require at least three linear regions to describe it, whereas NPrARX requires only two.

Convergence of ARX parameters for both states and the standard deviation (Std) over 100 iterations. The dashed lines represent ground truth values.

As shown in the convergence plots, the model parameters stabilize near the true values within approximately 40 iterations. The NPrARX model achieved a Parameter Fit ($F_\theta$) of 0.997 and a Mode Fit ($F_s$) of 0.996, indicating that the model correctly identified the active regime 99.6% of the time.

Learned nonlinear boundaries in the regressor space (u_{k-1} vs x_{k-1}). The black lines represent the boundaries discovered by the neural network gating function, successfully separating the two operating modes.

The neural network gate successfully discovered the hidden nonlinear switching logic, as illustrated in the regressor space partition. Computationally, the algorithm is efficient, averaging 0.42 seconds per iteration on a standard consumer laptop.

Conclusion

The NPrARX framework provides a flexible and accurate method for identifying hybrid systems with complex switching logic. By integrating neural gating, it overcomes the geometric rigidity of traditional piecewise models. Future research will focus on automating hyperparameter selection (such as the number of modes) and developing Bayesian extensions to provide uncertainty estimates, which are critical for safety-sensitive industrial applications.

Summary updated

Abstract

Hybrid system identification is a key tool to achieve reliable models of Cyber-Physical Systems from data. PieceWise Affine models guarantees universal approximation, local linearity and equivalence to other classes of hybrid system. Still, PWA identification is a challenging problem, requiring the concurrent solution of regression and classification tasks. In this work, we focus on the identification of PieceWise Auto Regressive with eXogenous input models with arbitrary regions (NPWARX), thus not restricted to polyhedral domains, and characterized by discontinuous maps. To this end, we propose a method based on a probabilistic mixture model, where the discrete state is represented through a multinomial distribution conditioned by the input regressors. The architecture is conceived following the Mixture of Expert concept, developed within the machine learning field. To achieve nonlinear partitioning, we parametrize the discriminant function using a neural network. Then, the parameters of both the ARX submodels and the classifier are concurrently estimated by maximizing the likelihood of the overall model using Expectation Maximization. The proposed method is demonstrated on a nonlinear piece-wise problem with discontinuous maps.
BibTeX
@misc{brusaferri2020identification,
  author = {Alessandro Brusaferri and Matteo Matteucci and Stefano Spinelli},
  title = {{Identification of Probability weighted ARX models with arbitrary domains}},
  year = {2020},
  eprint = {2009.13975},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2009.13975},
}
Figure