Particle Swarm Optimization-Based Model Abstraction and Explanation Generation for a Recurrent Neural Network

Yang Liu, Huadong Wang, Yan MaAlgorithms 2024 · DOI

TL;DR

This paper introduces a novel framework, PSO4RNN, that uses Particle Swarm Optimization (PSO) to create abstract models (Probabilistic Finite Automata - PFA) of complex Recurrent Neural Networks (RNNs) used in text classification. This approach significantly improves the explainability of RNNs by generating understandable insights into their decision-making processes, while also demonstrating effectiveness in adversarial text detection.

What Problem Does This Paper Address?

Recurrent Neural Networks (RNNs), particularly those used in text classification, are notoriously complex "black boxes." Their vast state spaces and intricate transition dynamics make it incredibly difficult to understand why they make specific predictions. This lack of explainability is a major hurdle for building trust in these models, debugging them, and ensuring their reliability, especially in security-sensitive applications like spam or harmful text detection. Existing methods for abstracting RNNs often suffer from low efficiency, significant information loss, or difficulty in handling large-scale models.

Methods

The core of the proposed solution, PSO4RNN, involves two main stages:

Stage I: Learning the Abstraction Model (PFA)

  1. State Trace Abstraction:
    • K-means Clustering: The first step is to partition the RNN's hidden states using k-means clustering. This reduces the complexity by grouping similar states, making the subsequent abstraction more manageable.
  2. Frequency Prefix Tree (FPT): Abstracted state traces are then organized into an FPT. This structure captures the sequential patterns of state transitions.
  3. PSO-based State Merging:
    • Particle Swarm Optimization (PSO): The FPT is then transformed into a Probabilistic Finite Automaton (PFA) using a modified PSO algorithm (PD-PSO). PSO is employed to optimize the merging of states and transitions within the FPT to create a PFA that balances similarity to the original RNN with low complexity. The fitness function for PSO is based on a combination of similarity and complexity, aiming to minimize the Bayesian Information Criterion (BIC).
  4. Partial Discrete PSO (PD-PSO): A variant of PSO is used to handle the discrete nature of some PFA components (like alphabet symbols) while maintaining the continuous processing of probability information.

Stage II: Explanation Generation

  • PFA-based Explanation: The learned PFA is then used to generate explanations. This involves identifying "cross-cluster," "cross-state," and "self-pointed" words based on their impact on state transitions.
  • Quantitative Scoring: The importance of words is quantified by calculating their impact on prediction probabilities (both presence and removal) and normalized to provide explanation scores.
  • Visualization: Explanations are visualized using word clouds and highlighted text, making them intuitive for users.

Datasets

The paper evaluates its approach on three types of datasets:

  • Regular Language: Tomita regular language classification dataset, used to validate learning effectiveness with artificial grammar rules.
  • Text Classification:
    • Logistics Reviews: A dataset of user reviews from Amazon e-commerce platforms.
  • Movie Reviews: The IMDB movie review dataset.
  • DLRV Dataset: A sentiment classification dataset extracted from customer evaluations of product logistics.

The paper notes that the movie review and logistics review datasets are publicly available.

Key Insights and Ideas

  • PSO for PFA Optimization: Using PSO to optimize the state merging process for PFA generation is a novel approach that effectively balances the trade-off between model complexity and similarity to the original RNN.
  • PFA as an Explainable Abstraction: PFAs, with their probabilistic nature, offer a more nuanced representation of RNN behavior than deterministic automata, leading to more accurate and informative explanations.
  • Hierarchical Explanation: The classification of words into "cross-cluster," "cross-state," and "self-pointed" provides a structured way to understand their influence on the RNN's decision-making, aligning with human intuition.
  • Adversarial Text Detection Utility: The PFA abstraction proves valuable not only for explanation but also for practical applications like detecting adversarial text attacks, demonstrating the robustness and utility of the learned models.

Key Results

  • Accuracy: PSO4RNN achieved significantly higher accuracy (80-95%) in abstracting RNN behavior compared to baseline methods (BSL1: 50-90%, BSL2: slightly above 90%).
  • Restoration Degree: PSO4RNN showed a superior restoration degree (0.67 for IMDB, 0.47 for DLRV, 0.83 for Tomita) compared to BSL1 and BSL2, indicating better generalization ability.
  • Efficiency: PSO4RNN demonstrated lower time consumption for learning abstraction models, with significantly fewer iterations and shorter training times, especially for larger datasets.
  • Complexity: The abstract models generated by PSO4RNN had fewer states compared to other methods, indicating a more compact and efficient representation.
  • Adversarial Text Detection: PSO4RNN achieved high detection accuracy (93% for Tomita, 92% for IMDB, 89% for DLRV) in adversarial text detection tasks.

Limitations and Future Work

  • Clustering Parameter Selection: The current method uses k-means for state partitioning, and the number of clusters (k) is selected through haphazard tests. This is not automatic and could be improved by integrating other clustering algorithms like DBSCAN or GMM.
  • Computational Efficiency: The state merging process can be computationally intensive, especially for very large datasets. Future work will focus on improving this through parallel or distributed learning strategies.
  • Runtime Abstraction: The paper suggests implementing runtime abstraction to adaptively compress state space, potentially using reinforcement learning for dynamic environments.
  • Generalizability: While the paper aims for universally applicable tasks, empirical studies were limited to two real-world datasets. Further research on larger-scale pre-trained models is needed.

Summary updated

Abstract

In text classifier models, the complexity of recurrent neural networks (RNNs) is very high because of the vast state space and uncertainty of transitions, which makes the RNN classifier’s explainability insufficient. It is almost impossible to explain the large-scale RNN directly. A feasible method is to generalize the rules undermining it, that is, model abstraction. To deal with the low efficiency and excessive information loss in existing model abstraction for RNNs, this work proposes a PSO (Particle Swarm Optimization)-based model abstraction and explanation generation method for RNNs. Firstly, the k-means clustering is applied to preliminarily partition the RNN decision process state. Secondly, a frequency prefix tree is constructed based on the traces, and a PSO algorithm is designed to implement state merging to address the problem of vast state space. Then, a PFA (probabilistic finite automata) is constructed to explain the RNN structure with preserving the origin RNN information as much as possible. Finally, the quantitative keywords are labeled as an explanation for classification results, which are automatically generated with the abstract model PFA. We demonstrate the feasibility and effectiveness of the proposed method in some cases.
BibTeX
@inproceedings{liu2024particle,
  author = {Yang Liu and Huadong Wang and Yan Ma},
  title = {{Particle Swarm Optimization-Based Model Abstraction and Explanation Generation for a Recurrent Neural Network}},
  booktitle = {Algorithms},
  year = {2024},
  doi = {10.3390/a17050210},
}
Figure