Detecting and Ranking Causal Anomalies in End-to-End Complex System

Ching Chang, Cheng-Han Yeh, Wen-Chih Peng

TL;DR

This paper introduces RCAE2E, a novel framework for detecting and ranking causal anomalies in complex industrial systems. RCAE2E addresses limitations of prior methods by explicitly accounting for diverse operational states and time-lagged correlations, leading to more accurate anomaly root cause identification.

What Problem Does This Paper Address?

Modern large-scale factories rely heavily on automated monitoring systems to detect anomalies using vast amounts of sensor data. While detecting anomalies is important, the true value lies in identifying their root causes. Existing methods, like the popular RCA (Ranking Causal Anomalies) approach, often fall short. RCA typically models a machine's behavior using a single, time-invariant correlation network. This approach has two major drawbacks:

  1. Ignores State Diversity: Real-world systems operate in multiple distinct states (e.g., different production modes). A single model cannot accurately capture the correlations within each of these states.
  2. Lacks Time-Lagged Correlation Handling: Correlations between sensors can manifest with different delays. Prior methods struggle to separately consider these time-lagged relationships, which are crucial for understanding how faults propagate.

The authors propose RCAE2E to overcome these limitations, aiming to provide a more robust and accurate way to pinpoint the origin of system failures.

Methods

RCAE2E is built upon two core novel methods:

  1. TICC_GTC (Toeplitz Inverse Covariance-based Clustering with Global Temporal Consistency): This method is used for creating system profiles. It's an extension of the TICC algorithm, which segments multivariate time series data into distinct states (clusters), each represented by a Markov Random Field (MRF). TICC_GTC enhances TICC by introducing global temporal consistency. This constraint ensures that cluster assignments for data subsequences from adjacent runs (at the same timestamp) are aligned, reflecting the inherent regularity in industrial processes. This addresses the "diversity of states" problem.
  2. RCA_SCC (Ranking Causal Anomalies with Separate Consideration of Correlations): This method is responsible for identifying causal anomalies. It's an adaptation of the RCA approach but specifically designed to separately consider correlations with different time-lags. This allows for a more nuanced understanding of fault propagation through the system. RCA_SCC takes the profiles generated by TICC_GTC and uses them to identify anomalies.

The overall RCAE2E framework involves using TICC_GTC to build a profile of normal system behavior and then employing RCA_SCC to detect deviations from this profile, ranking them by their causal impact.

Datasets

The paper evaluates RCAE2E on two types of data:

  1. Synthetic Data: Generated based on synthetic profiles (MRFs and cluster assignments) to control experimental conditions and verify the method's core functionalities.
  2. Real-World Large-Scale Photoelectric Factory Data: This dataset consists of machine sensor data from a factory, providing a realistic testbed for the proposed framework. The data description includes: 136 runs, 1460 timestamps per run, and 48 sensors per machine.

Key Insights and Ideas

  • State Diversity is Crucial: The experiments strongly suggest that ignoring distinct operational states leads to significantly poorer anomaly detection performance. A single, time-invariant model is insufficient for complex systems.
  • Time-Lagged Correlations Matter: Explicitly modeling correlations that occur with different delays is essential for accurately tracing the propagation of faults. RCAE2E's ability to handle this separately improves its robustness.
  • Global Temporal Consistency: For industrial settings where production is generally regular, enforcing consistency across adjacent runs (at the same timestamp) is a valuable constraint for building more reliable system profiles.
  • Causal Ranking is Key: The ultimate goal is not just to detect an anomaly but to understand its origin. RCAE2E's ranking mechanism helps prioritize anomalies based on their likely causal impact.

Key Results

The experimental results demonstrate the effectiveness of RCAE2E:

  • Synthetic Data: Figures 5 and 6 show that RCAE2E significantly outperforms other methods (RCA, gRank, mRank, LBP) when considering multiple states and varying lengths of cross-time correlations, respectively. Figure 7 indicates good noise robustness. Figure 8 shows the monotonic decrease in the objective function during updates, demonstrating convergence.
  • Real-World Data: Figure 9 shows that RCAE2E achieves superior performance in precision, recall, and nDCG compared to other methods on real factory data. Figure 10 further highlights that even when RCA is combined with various clustering methods, RCAE2E still provides better results, underscoring the benefit of its integrated approach.

Limitations and Future Work

  • Limitations: The paper acknowledges that RCA's assumption of fault propagation (i.e., faults rarely being isolated) is a core tenet. While this is generally true in industrial systems, extremely rare, isolated faults might be harder to detect or rank accurately. The computational complexity of the algorithms, particularly for very large-scale systems with many sensors and long time series, could be a practical concern.
  • Future Work: The paper doesn't explicitly detail future work, but the success of RCAE2E suggests potential applications in other complex dynamic systems beyond industrial manufacturing. Further research could focus on optimizing computational efficiency, exploring different anomaly generation strategies, and extending the framework to handle more complex fault types.

Summary updated

Abstract

With the rapid development of technology, the automated monitoring systems of large-scale factories are becoming more and more important. By collecting a large amount of machine sensor data, we can have many ways to find anomalies. We believe that the real core value of an automated monitoring system is to identify and track the cause of the problem. The most famous method for finding causal anomalies is RCA, but there are many problems that cannot be ignored. They used the AutoRegressive eXogenous (ARX) model to create a time-invariant correlation network as a machine profile, and then use this profile to track the causal anomalies by means of a method called fault propagation. There are two major problems in describing the behavior of a machine by using the correlation network established by ARX: (1) It does not take into account the diversity of states (2) It does not separately consider the correlations with different time-lag. Based on these problems, we propose a framework called Ranking Causal Anomalies in End-to-End System (RCAE2E), which completely solves the problems mentioned above. In the experimental part, we use synthetic data and real-world large-scale photoelectric factory data to verify the correctness and existence of our method hypothesis.
BibTeX
@misc{chang2023detecting,
  author = {Ching Chang and Cheng-Han Yeh and Wen-Chih Peng},
  title = {{Detecting and Ranking Causal Anomalies in End-to-End Complex System}},
  year = {2023},
  eprint = {2301.07281},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2301.07281},
}
Figure