Influence Functions for Sequence Tagging Models

Sarthak Jain, Varun Manjunatha, Byron C. Wallace, Ani Nenkova

Abstract

Interpretability in machine learning often relies on influence functions to trace model predictions back to specific training instances. However, in sequence tagging tasks like Named Entity Recognition (NER), standard instance-level influence is often too coarse to identify the precise source of errors. This paper introduces segment influence, a method designed to pinpoint the specific snippets within training sequences that drive a model's predictions. By adapting influence functions for Conditional Random Fields (CRFs) and implementing computational optimizations, the authors demonstrate that segment-level analysis significantly outperforms traditional methods in debugging systematic annotation errors.

Introduction

Understanding why a model makes a specific prediction is critical for debugging and safety. Influence functions provide a mathematical framework for this by estimating how a model's parameters—and thus its predictions—would change if a specific training example were modified or removed. While effective for simple classification, these methods struggle with structured prediction tasks like sequence tagging. In NER, for instance, a model might misclassify "Manchester United" as a Location because it has seen "Manchester" labeled as a Location in other contexts. An instance-level influence function might point to a 50-word sentence containing "Manchester," but it cannot isolate the specific phrase causing the confusion. This paper addresses this limitation by proposing a granular, segment-level approach.

Methodology

Defining Segment Influence

The authors define segment influence as the change in the loss of a specific test segment (e.g., a single entity) resulting from a perturbation of a specific segment in a training instance. Because retraining a model for every possible training segment is computationally prohibitive, the authors utilize a first-order Taylor expansion to approximate this change.

Computational Efficiency

Applying influence functions to modern architectures like BERT or BigBird presents significant scaling challenges. To maintain tractability, the authors introduce several key optimizations:

  1. Layer-wise Gradients: Instead of calculating gradients for the entire transformer backbone, the method focuses on the final classification and transition layers ($W$ and $T$) of the CRF.
  2. Hessian Simplification: The authors found that the Hessian term, which accounts for parameter interactions, can often be simplified or ignored without significantly degrading the ranking of influential examples. This drastically reduces the memory and time requirements.
  3. Search Constraints: To manage complexity, the current implementation focuses on single-token segments, reducing the search space from $O(T^2)$ to $O(T)$.

Experimental Evaluation

Datasets

The method was evaluated on two distinct NER benchmarks:

  • CoNLL-2003: A standard news-based dataset used to investigate systematic errors, such as the misclassification of sports teams (Organizations) that share names with cities (Locations).
  • EBM-NLP: A medical abstract dataset characterized by inconsistent annotations regarding drug dosages. Some entries include dosages within the "intervention" entity, while others exclude them, creating a "noisy" training environment.

Key Results

The proposed segment influence method demonstrated high fidelity and superior debugging capabilities:

  • Validation: The approximation achieved a Pearson correlation of 0.89 with the "exact" influence calculated through exhaustive retraining.
  • Error Detection: In the EBM-NLP dosage inconsistency task, segment influence identified the conflicting training segments 97.9% of the time. In comparison, standard instance-level influence only succeeded in 35.4% of cases.
  • Artifact Retrieval: In synthetic tests where specific tokens were inserted to intentionally trigger errors, the method perfectly identified those tokens as the primary source of influence.

Discussion and Future Work

Granularity and Systematic Noise

The primary advantage of segment influence is its ability to handle "systematic" noise. Unlike random noise, systematic noise occurs when a model learns a flawed but consistent rule (e.g., "all city names are Locations"). By pinpointing the specific tokens reinforcing these rules, developers can more effectively clean training data and improve model robustness.

Limitations

Despite its success, the method faces challenges related to the non-convexity of the loss surface in deep learning. Removing a segment changes the marginal likelihood objective, which may lead to different local minima during retraining, potentially affecting the accuracy of the influence approximation. Furthermore, while optimized, the computational cost remains high for very large datasets.

Conclusion

Segment influence represents a significant step forward in making sequence tagging models more transparent. By moving beyond the instance level, this approach provides the precision necessary to debug complex, structured models. Future research will focus on extending this framework to generative sequence-to-sequence models, such as Large Language Models (LLMs), where the challenges of massive vocabulary sizes and autoregressive decoding present new hurdles for influence-based interpretability.

Summary updated

Abstract

Many language tasks (e.g., Named Entity Recognition, Part-of-Speech tagging, and Semantic Role Labeling) are naturally framed as sequence tagging problems. However, there has been comparatively little work on interpretability methods for sequence tagging models. In this paper, we extend influence functions - which aim to trace predictions back to the training points that informed them - to sequence tagging tasks. We define the influence of a training instance segment as the effect that perturbing the labels within this segment has on a test segment level prediction. We provide an efficient approximation to compute this, and show that it tracks with the true segment influence, measured empirically. We show the practical utility of segment influence by using the method to identify systematic annotation errors in two named entity recognition corpora. Code to reproduce our results is available at https://github.com/successar/Segment_Influence_Functions.
BibTeX
@inproceedings{jain2022influence,
  author = {Sarthak Jain and Varun Manjunatha and Byron C. Wallace and Ani Nenkova},
  title = {{Influence Functions for Sequence Tagging Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  doi = {10.18653/v1/2022.findings-emnlp.58},
}
Figure