Second-Order Component Analysis for Fault Detection

Jingchao Peng, Haitao Zhao, Zhengwei Hu

Abstract

Detecting equipment failure in complex industrial environments is challenging due to nonlinear data relationships, high dimensionality, and heteroscedastic noise. This paper presents Second-Order Component Analysis (SCA), a method that combines high-order neural network mapping with orthogonal constraints. By utilizing a second-order expansion in the encoder and an orthogonal decoder optimized on a Riemannian manifold, SCA provides robust feature extraction and superior fault detection performance compared to traditional linear and nonlinear techniques.

Introduction

In industrial systems such as chemical plants or semiconductor manufacturing, early fault detection is vital for operational safety and cost efficiency. However, sensor data often exhibits three difficult characteristics: nonlinearity, high correlation (the "curse of dimensionality"), and heteroscedasticity (varying noise variance). While Principal Component Analysis (PCA) offers structural robustness through orthogonality, it lacks the capacity to model nonlinearities. Conversely, standard Autoencoders (AE) capture nonlinearities but are prone to overfitting by learning noise as signal.

Methodology: Second-Order Component Analysis (SCA)

SCA bridges the gap between PCA and AE by introducing a "shallow" yet "wide" autoencoder architecture with two primary innovations.

Second-Order Encoder

Standard encoders process raw input vectors directly. SCA's encoder incorporates a second-order expansion, including squared terms ($x^2$) and cross-products ($x_i x_j$). This mathematical expansion allows the model to explicitly account for heteroscedasticity, which is often a critical factor in distinguishing normal operational states from anomalies.

The second-order expansion term used in the SCA encoder to handle heteroscedastic data.

Orthogonal Decoder and Manifold Optimization

To prevent overfitting and ensure the extraction of non-redundant features, the SCA decoder is constrained to be orthogonal ($W^T W = I$). This constraint forces the network to focus on the most significant subspace, effectively filtering out noise.

Because orthogonality is a hard constraint that standard gradient descent cannot easily handle, the model is trained using a Geometric Conjugate Gradient algorithm. This approach optimizes the weights on a product manifold (a Stiefel manifold for the orthogonal weights and a Euclidean manifold for others), ensuring that the structural constraints are strictly maintained throughout training.

Experimental Evaluation

The performance of SCA was evaluated using two datasets:

  1. Toy Heteroscedastic Process: A synthetic dataset designed to test the necessity of second-order terms in separating data with non-constant variance.
  2. Tennessee-Eastman Process (TEP): A standard industrial benchmark simulating a chemical plant with 52 variables and 21 distinct fault modes.

Results and Discussion

SCA demonstrated superior detection capabilities across the 21 TEP faults, achieving the lowest Missed Detection Rate in 7 cases, outperforming PCA (5), KPCA (4), and standard AE (4). Notably, SCA significantly reduced the False Alarm Rate (FAR), often reaching 0.00% in scenarios where other nonlinear methods struggled with overfitting.

Visualization of Feature Spaces

The effectiveness of SCA is most apparent in the visualization of the extracted features. While PCA and AE often produce overlapping clusters of normal and faulty data, SCA creates tight clusters for normal data and clear separation for anomalies, as shown in the comparative plots below.

Visualization plots comparing PCA, KPCA, AE, and SCA. SCA shows the clearest separation between normal and fault data.

Conclusion

Second-Order Component Analysis offers a theoretically grounded and practically effective solution for industrial fault detection. By integrating second-order mapping with hard orthogonal constraints via manifold optimization, it achieves a balance between nonlinear flexibility and structural robustness. Future work will explore deep variants of SCA and methods to reduce the computational overhead associated with manifold-based optimization.

Summary updated

Abstract

Process monitoring based on neural networks is getting more and more attention. Compared with classical neural networks, high-order neural networks have natural advantages in dealing with heteroscedastic data. However, high-order neural networks might bring the risk of overfitting and learning both the key information from original data and noises or anomalies. Orthogonal constraints can greatly reduce correlations between extracted features, thereby reducing the overfitting risk. This paper proposes a novel fault detection method called second-order component analysis (SCA). SCA rules out the heteroscedasticity of pro-cess data by optimizing a second-order autoencoder with orthogonal constraints. In order to deal with this constrained optimization problem, a geometric conjugate gradient algorithm is adopted in this paper, which performs geometric optimization on the combination of Stiefel manifold and Euclidean manifold. Extensive experiments on the Tennessee-Eastman benchmark pro-cess show that SCA outperforms PCA, KPCA, and autoencoder in missed detection rate (MDR) and false alarm rate (FAR).
BibTeX
@article{peng2021second,
  author = {Jingchao Peng and Haitao Zhao and Zhengwei Hu},
  title = {{Second-Order Component Analysis for Fault Detection}},
  journal = {Journal of Process Control},
  year = {2021},
  doi = {10.1016/j.jprocont.2021.10.011},
}
Figure