RI2AP: Robust and Interpretable 2D Anomaly Prediction in Assembly Pipelines

Chathurangi Shyalika, Kaushik Roy, Renjith Prasad, Fadi El Kalach, Yuxin Zi, Priya Mittal, Vignesh Narayanan, Ramy F. Harik, Amit P. Sheth2024 · DOI

TL;DR

This paper introduces RI2AP, a novel method for robust and interpretable anomaly prediction in manufacturing assembly lines. By modeling anomalies in two dimensions (when they occur and their dependencies) and incorporating a causal-influence framework for interpretability, RI2AP significantly improves prediction accuracy, outperforming existing methods by up to 30% in F1 score.

What Problem Does This Paper Address?

Predicting anomalies in manufacturing assembly lines is crucial for efficiency and cost reduction, especially in complex processes like rocket assembly where premature part failures can be costly. However, current machine learning methods struggle with several challenges:

  • Data Imbalance: Anomalies are rare events, leading to imbalanced datasets that are difficult for models to learn from.
  • Data Scarcity: High-fidelity simulation data for real-world processes like rocket assembly is expensive and difficult to obtain.
  • Complex Dependencies: Anomalies can be interdependent, and traditional methods often fail to capture these relationships.
  • Lack of Interpretability: Existing models often provide black-box predictions, making it hard for domain experts to understand why an anomaly is predicted and how to prevent it.

This paper aims to address these limitations by developing a method that can accurately predict anomalies, understand their interdependencies, and provide interpretable insights to domain experts.

Methods

The core of RI2AP lies in its novel approach to anomaly prediction:

  • 2D Anomaly Prediction: Instead of just predicting if an anomaly will occur, RI2AP predicts both the future sensor values and a "compositional anomaly" token. This token represents the severity and type of anomaly, encoded in a monotonically increasing manner to reflect the spatial structure of rocket assembly (e.g., bottom parts detaching before top parts).
  • Robustness to Rarity: The method uses a regression objective, which helps overcome the challenges posed by rare anomalous events without requiring special sampling strategies.
  • Interpretable Anomaly Modeling: RI2AP employs a causal-influence framework, specifically using combining rules (like Noisy-OR and Noisy-MAX), to model how individual sensor measurements influence anomaly predictions. This allows domain experts to understand the contribution of each sensor to a prediction, offering insights beyond traditional "black-box" explainable AI methods.
  • Function Approximators: The paper explores several function approximators for modeling individual sensor measurements, including Long Short-Term Memory Networks (LSTMs), Transformer architectures (decoder-only), and a "Method of Moments" approach. The Method of Moments is highlighted for its robustness to noise and class imbalance.

Datasets

The primary dataset used in this research is the Future Factories (FF) dataset. This dataset originates from a simulation of a rocket assembly pipeline, designed to adhere to industrial standards. It includes measurements from various sensors and control mechanisms. The authors selected 20 out of the original 41 measurements that contained high information content. Anomalies in the dataset are categorized into five distinct groups: "None" (normal), "Type 1" (one rocket part missing), "Type 2" (two rocket parts missing), "Type 3" (three rocket parts missing), and "Type 4" (miscellaneous anomalies).

Key Insights and Ideas

  • Compositional Anomaly Encoding: Representing anomalies not just as discrete classes but as compositional entities with increasing severity (e.g., based on the number of parts failing) is a key innovation that captures the underlying physics of assembly processes.
  • Causal-Influence for Interpretability: Moving beyond standard XAI techniques, the use of combining rules from causal inference provides a more natural and domain-expert-friendly way to understand why a prediction is made, directly linking sensor readings to potential anomalies.
  • Method of Moments as a Robust Predictor: The "Method of Moments" approach, particularly when combined with neural networks, shows significant promise in handling the inherent noise and class imbalance in manufacturing data, outperforming other function approximators.
  • 2D Prediction for Enhanced Understanding: Predicting both sensor values and anomaly tokens simultaneously provides a richer output that can better capture the dynamics of assembly processes.

Key Results

  • RI2AP achieved up to a 30-point improvement in F1 measure compared to current ML methods on the rocket assembly simulation.
  • The Method of Moments function approximator consistently outperformed LSTM and Transformer models, especially in predicting rare anomalies.
  • The Noisy-OR combining rule generally yielded better results than Noisy-MAX, suggesting that compounded failures are more indicative of anomalies than single critical part failures in this context.
  • The model was successfully deployed in a real manufacturing setting (Future Factories cell), demonstrating its practical applicability.

Limitations and Future Work

  • Current Implementation: The current RI2AP implementation uses individual function approximators for each sensor. Future work includes developing a multisensor function approximator that considers all sensors simultaneously to potentially further improve accuracy and quantify the performance gap.
  • Combining Rules: The authors plan to investigate alternative combining rules and explore the use of do-calculus to manage potential backdoors and confounding factors.
  • Interpretability Enhancement: Future work aims to develop higher-level representations of causal phenomena related to anomalies to further enhance interpretability for domain experts.
  • Deployment Challenges: During deployment, challenges included adapting code to data formats, sensor-related issues, and accurately simulating anomalies. These were addressed through expert intervention and iterative refinement.

Summary updated

BibTeX
@misc{shyalika2024ri,
  author = {Chathurangi Shyalika and Kaushik Roy and Renjith Prasad and Fadi El Kalach and Yuxin Zi and Priya Mittal and Vignesh Narayanan and Ramy F. Harik and Amit P. Sheth},
  title = {{RI2AP: Robust and Interpretable 2D Anomaly Prediction in Assembly Pipelines}},
  year = {2024},
  doi = {10.3390/s24103244},
}
Figure