RNNRepair: Automatic RNN Repair via Model-based Analysis

Xiaofei Xie, Wenbo Guo, Lei Ma, Wei Le, Jian Wang, Lingjun Zhou, Yang Liu, Xinyu XingICML 2021 · OpenReview

Introduction

RNNRepair is a lightweight, model-based framework designed to debug and fix Recurrent Neural Networks (RNNs) without the computational nightmare of gradient-based influence functions. By clustering the internal hidden states of an RNN into a simplified "influence model," the tool can pinpoint exactly which training samples—or even which specific segments of a sequence—caused a misclassification. It then "repairs" the model through targeted data augmentation.

The Challenge with RNN Debugging

Deep learning models are notoriously "black boxes," and RNNs are particularly opaque due to their stateful nature and sequential dependencies. When an RNN makes a mistake, engineers face two major hurdles:

  1. Attribution: Which training samples are "responsible" for this wrong prediction?
  2. Repair: How can we fix this specific error without breaking the rest of the model?

Existing techniques, like Influence Functions, try to solve this by calculating how much a model's loss would change if a specific training point were removed. However, these rely on gradients, which in RNNs often suffer from "vanishing" or "exploding" issues. Furthermore, standard methods treat the entire input as one unit, making it impossible to identify if a specific part of a sequence (like a single offensive word in a long sentence) is the root cause of the failure.

RNNRepair: A Model-Based Approach

The authors move away from gradient calculus and instead use Semantic-guided State Abstraction to understand model behavior.

State Clustering and the Influence Model

They collect the hidden state vectors produced by the RNN across all training data and use Gaussian Mixture Models (GMM) to cluster them into "abstract states." This turns the complex RNN into a simplified graph where transitions between abstract states are mapped. They define a Confidence Score for each state, measuring how consistently a state leads to a specific prediction.

For any input, they extract a "trace" of these abstract states. By comparing the trace of a failed test input to the traces of training inputs, they can calculate an influence score to find the most relevant training samples.

Trace of abstract states over time for a sequential MNIST input

Surgical Repair Mechanisms

Once the influential elements are identified, RNNRepair applies targeted fixes:

  • Sample-level Repair: If a whole image (like a messy "4") is misclassified, the system identifies the most influential training samples of the correct class. It then uses data augmentation (e.g., rotation, shearing) to create new, highly influential samples that "nudge" the decision boundary in the right direction.
  • Segment-level Repair: If a specific word (e.g., "heck") causes a sentiment analyzer to flip from positive to negative, the system identifies that segment and inserts it into other positive training samples. This teaches the model that the word doesn't necessarily imply a negative sentiment.
Conceptual diagram of decision boundary shifting and examples of augmented MNIST digits

Experimental Results

The researchers evaluated RNNRepair on three primary datasets: MNIST (treated as a sequential task where the RNN reads the image row-by-row), the Toxic Comment Dataset, and the Standard Sentiment Treebank (SST).

Key Insights and Performance

  • Efficiency is a Feature: Because it avoids intensive gradient computation, RNNRepair is orders of magnitude faster than state-of-the-art methods. The influence analysis took an average of 1.16 seconds per error, compared to 70.13 seconds for K&L and over 5,600 seconds for SGD-based methods.
  • Mislabeled Data Detection: RNNRepair was significantly more effective at identifying "influential flips"—mislabeled training data that actually caused test errors—than baseline methods.
  • Successful Repairs: On the MNIST dataset, RNNRepair successfully fixed 50.9% of consistently failing inputs by adding only 161 targeted synthetic samples. In contrast, a random augmentation strategy only fixed 18.7%.
  • NLP Precision: On the Toxic and SST datasets, the segment-level repair consistently outperformed random insertion strategies, proving that targeting the "root cause" segment is vital for effective model editing.

Limitations and Future Work

The authors acknowledge that their GMM-based partitioning is currently somewhat simple. More fine-grained refinement of how states are clustered could lead to even more accurate influence models.

Additionally, the current repair mechanism focuses heavily on "positive-to-negative" type errors (where a specific segment triggers a wrong class). Future work is needed to extend this to more complex error types, such as "negative-to-positive" errors where the absence of a feature or more complex semantic context is the problem. They also plan to explore more diverse data augmentation techniques, such as using Generative Adversarial Networks (GANs) to create even more "influential" repair samples.

Summary updated

Abstract

Deep neural networks are vulnerable to adversarial attacks. Due to their black-box nature, it is rather challenging to interpret and properly repair these incorrect behaviors. This paper focuses on...
BibTeX
@inproceedings{xie2021rnnrepair,
  author = {Xiaofei Xie and Wenbo Guo and Lei Ma and Wei Le and Jian Wang and Lingjun Zhou and Yang Liu and Xinyu Xing},
  title = {{RNNRepair: Automatic RNN Repair via Model-based Analysis}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
  url = {https://openreview.net/forum?id=OOL1ZjsyaG3},
}
Figure