GEval: Tool for Debugging NLP Datasets and Models

Filip Graliński, Anna Wróblewska, Tomasz Stanisławek, Kamil Grabowski, Tomasz GóreckiACL 2019

TL;DR

This paper introduces GEval, a novel "no-box" debugging tool for machine learning models, particularly effective for NLP tasks. GEval uses statistical hypothesis testing to identify specific features (like words or phrases) that negatively impact model performance, offering a more efficient and interpretable alternative to existing methods for large datasets.

What Problem Does This Paper Address?

As machine learning models, especially deep learning architectures, become more complex and are trained on massive datasets, understanding why they make certain predictions or fail in specific instances becomes increasingly challenging. This "black box" nature hinders debugging, model improvement, and trust. While interpretability research is growing, existing methods like LIME can be computationally expensive for large datasets, and white-box methods require deep access to model internals, which isn't always feasible. The authors aim to bridge this gap by providing a fast, model-agnostic tool that pinpoints problematic features in datasets and models, enabling targeted improvements.

Methods

GEval employs a "no-box" debugging approach, meaning it doesn't require access to the model's internal workings. Instead, it relies on the input data, the model's outputs, and the ground truth labels. The core idea is to treat the evaluation metric (e.g., accuracy, F1-score, BLEU) not just as a single score for the entire dataset, but as an item-wise score.

The process involves:

  1. Item-wise Evaluation: Calculating the evaluation metric for each individual data point.
  2. Feature Extraction: Identifying potential "features" that might be present in the data. These can be simple tokens, bigrams, word shapes, or even combinations of features.
  3. Hypothesis Testing: Using the Mann-Whitney U test (or Kendall's Tau, which is equivalent for binary features) to compare the distribution of item-wise evaluation scores for data points with a specific feature against those without it.
  4. P-value Calculation: The p-value from this test indicates the probability that the observed difference in scores is due to chance. A low p-value suggests the feature significantly impacts the model's performance.
  5. Ranking Features: GEval ranks features based on their p-values, highlighting those that most negatively influence the evaluation metric.

This approach allows for the identification of "worsening features" – those that, when present, lead to a significant drop in performance.

Datasets

The paper demonstrates GEval's utility on several standard NLP datasets and tasks:

  • Twitter sentiment analysis: Using the ULMFiT library for text classification.
  • German-to-English machine translation: Comparing LIUM and UEDin models on the WMT-2017 dataset.
  • Named entity recognition (NER): Using the FLAIR model on the CoNLL 2003 dataset.

The authors do not introduce new datasets but rather apply their tool to existing benchmarks to showcase its effectiveness.

Key Insights and Ideas

  • "No-box" debugging is feasible and powerful: GEval proves that deep model introspection isn't always necessary to find critical issues. Access to inputs, outputs, and labels is sufficient.
  • Item-wise metrics are key: Breaking down overall performance into per-item scores unlocks the ability to correlate specific features with performance drops.
  • Statistical significance for feature impact: Using p-values from hypothesis tests provides a principled way to identify features that genuinely affect model performance, rather than just those that appear frequently.
  • Efficiency for large datasets: GEval's approach is significantly faster than perturbation-based methods like LIME, making it practical for large-scale NLP applications.
  • Actionable insights: The identified features are often directly interpretable (e.g., specific words, negations, capitalization patterns), guiding concrete steps for data augmentation, preprocessing, or model adjustments.

Key Results

  • Sentiment Analysis: GEval identified features like "though," "wouldn't," and "can't" as negatively impacting sentiment classification accuracy. The paper shows that training a model specifically on "though" examples improved overall accuracy.
  • Machine Translation: GEval highlighted specific translation issues, such as the incorrect translation of "euro" to "pounds," and problematic words like "be," "people," and "Menschen" that vary in meaning across contexts.
  • Named Entity Recognition: The tool pinpointed issues related to uppercase text, entities at the beginning of sentences, the "MISC" class, and the mixing of entity types (e.g., LOC vs. ORG). It also identified difficulties with numbers and country names used as adjectives.
  • Efficiency: GEval processed very large datasets in minutes, a stark contrast to LIME's per-data-point processing time.

Limitations and Future Work

  • P-value interpretation: The authors acknowledge the common criticisms of p-values but argue they are used here solely for ranking features, not for strict hypothesis testing with significance levels.
  • Metric dependency: While GEval is designed to be metric-agnostic, the paper notes that for some metrics, simple differences might not be the most sound way to compare scores, suggesting alternative comparison methods might be needed.
  • Feature engineering: The paper discusses various ways to generate features, but the effectiveness of GEval can depend on the quality and relevance of the extracted features. Future work could explore more sophisticated automatic feature generation.
  • "No-box" vs. "Black-box": The authors clarify that their method is "no-box" rather than strictly "black-box" because it doesn't re-run the model, but rather analyzes existing outputs.

The authors suggest that GEval's flexibility allows for its application to any evaluation metric and any type of data point, opening avenues for its use in diverse ML domains beyond NLP.

Summary updated

BibTeX
@inproceedings{gralinski2019geval,
  author = {Filip Graliński and Anna Wróblewska and Tomasz Stanisławek and Kamil Grabowski and Tomasz Górecki},
  title = {{GEval: Tool for Debugging NLP Datasets and Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure