Abstract
Hybrid systems, which switch between different linear behaviors based on operating conditions, are prevalent in cyber-physical systems. Traditional PieceWise AutoRegressive with eXogenous input (PWARX) models often struggle with nonlinear boundaries between these regimes, requiring redundant sub-models to approximate complex transitions. This paper proposes the Nonlinearly Probabilistic ARX (NPrARX) framework, which utilizes a neural network gating function within a Mixture of Experts architecture to learn arbitrary, nonlinear boundaries directly from data. Using an Expectation-Maximization (EM) algorithm, the framework concurrently identifies system dynamics and switching logic. Experimental results on a complex synthetic benchmark demonstrate high precision in both parameter estimation and mode identification.
Introduction
Many real-world "Cyber-Physical Systems," such as robotic arms hitting physical limits or chemical plants switching valves, exhibit hybrid dynamics. These systems behave linearly for periods before suddenly shifting to a different regime based on specific conditions. The standard tool for modeling such behavior is the PWARX model. However, traditional PWARX identification assumes that the boundaries between behaviors are polyhedral—essentially straight lines or flat planes.
In practice, these boundaries are often nonlinear. Forcing linear partitions onto curved boundaries leads to model over-complexity, as extra sub-models must be invented just to approximate the curves. The NPrARX framework addresses this by allowing the model to learn curved, arbitrary boundaries directly from data, leading to more parsimonious and interpretable models.
Methodology
The NPrARX approach treats system identification as a Mixture of Experts (MoE) task consisting of two primary components:
- The Experts: These are standard ARX sub-models (linear equations) that describe the system's dynamics within specific regions.
- The Gate: This component determines which expert is active. By using a Feed-Forward Neural Network with tanh activation functions as the gating mechanism, the model can draw complex, curved boundaries in the regressor space.
Training is performed using the Expectation-Maximization (EM) algorithm. In the E-step, the model calculates the posterior probability that each expert was responsible for a specific output. In the M-step, the ARX parameters are updated via weighted least squares, and the neural network weights are optimized using the Adam optimizer to match the "soft labels" generated in the E-step. To avoid local minima where experts model the same average behavior, the authors employ high-variance random initialization for the neural network weights.
Experimental Results
The framework was evaluated using a synthetic benchmark designed to test hybrid identification under non-linearly separable conditions. The system consists of two distinct affine maps arranged such that a standard PWARX model would require at least three linear regions to describe it, whereas NPrARX requires only two.
As shown in the convergence plots, the model parameters stabilize near the true values within approximately 40 iterations. The NPrARX model achieved a Parameter Fit ($F_\theta$) of 0.997 and a Mode Fit ($F_s$) of 0.996, indicating that the model correctly identified the active regime 99.6% of the time.
The neural network gate successfully discovered the hidden nonlinear switching logic, as illustrated in the regressor space partition. Computationally, the algorithm is efficient, averaging 0.42 seconds per iteration on a standard consumer laptop.
Conclusion
The NPrARX framework provides a flexible and accurate method for identifying hybrid systems with complex switching logic. By integrating neural gating, it overcomes the geometric rigidity of traditional piecewise models. Future research will focus on automating hyperparameter selection (such as the number of modes) and developing Bayesian extensions to provide uncertainty estimates, which are critical for safety-sensitive industrial applications.
Summary updated