Azimuth: Systematic Error Analysis for Text Classification

Gabrielle Gauthier-melancon, Orlando Marquez Ayala, Lindsay Brin, Chris Tyler, Frederic Branchaud-charron, Joseph Marinier, Karine Grande, Di LeEMNLP 2022

TL;DR

This paper introduces Azimuth, an open-source tool designed to systematically analyze errors in text classification models. It addresses the current lack of mature tooling for error analysis, which is crucial for building reliable AI systems, by providing a unified platform for dataset analysis and model quality assessment.

What Problem Does This Paper Address?

Developing reliable and trustworthy AI systems, especially in enterprises, is challenging because unlike traditional software, isolating the sources of errors in AI models is difficult. NLP models can suffer from various issues like poor linguistic capabilities, hallucinations, learning spurious correlations from data artifacts, or amplifying social biases. While model training and hyperparameter tuning have well-established tools and processes, the error analysis stage is less mature. Current practices often rely on custom scripts and intuition, making them informal, ad hoc, and time-consuming, which can lead practitioners to skip or rush this critical step. This hinders traceability, accountability, and the ability to identify and fix failure modes, ultimately risking the deployment of problematic models that can lead to legal penalties, financial losses, and user distrust.

Methods

Azimuth proposes a systematic error analysis process that combines dataset analysis and model quality assessment.

Dataset Analysis focuses on understanding and improving the data used for training and evaluation. It identifies four common data problems:

  • Data Shift: Differences between training and validation splits.
  • Class Imbalance: Uneven distribution of examples across classes.
  • Class Definition: Ambiguous or overlapping class definitions.
  • Problematic Examples: Examples that are outliers or difficult to label.

Model Quality Assessment examines model behavior and generalization. It goes beyond high-level metrics and includes four types of evaluation:

  • Data Subpopulations: Analyzing model performance on specific subsets of data (e.g., long sentences, specific labels).
  • Annotation Artifacts Discovery: Identifying patterns in labeled data that models exploit instead of learning the task, often using saliency maps.
  • Behavioral Testing: Assessing model robustness by perturbing inputs and observing prediction changes.
  • Uncertainty-based Analysis: Examining model confidence and identifying examples where the model is uncertain.

Azimuth integrates these techniques into a user-friendly, open-source tool.

Datasets

The paper uses the CLINC-OOS dataset for their case study. This is a large intent classification dataset with 150 "in-scope" classes and one "Out-of-Scope" (OOS) class. They also mention using a DistilBert model trained on this dataset.

Key Insights and Ideas

  • Error analysis is a critical, yet underdeveloped, stage: The paper highlights that while model training is well-resourced, error analysis is often informal and ad hoc, leading to significant risks.
  • A unified tool is needed: Azimuth aims to consolidate various error analysis techniques into a single, accessible platform, reducing the need for custom scripts.
  • Dataset quality is paramount: The framework emphasizes that understanding and improving the dataset is as crucial as assessing model performance. Issues like class imbalance and definition directly impact model behavior.
  • High-level metrics can be misleading: The paper stresses that metrics like accuracy can hide critical failure modes and biases, necessitating deeper analysis.
  • Smart tags are powerful for discovery: Azimuth uses "smart tags" to automatically flag problematic examples based on various analyses (e.g., high epistemic uncertainty, conflicting neighbors), guiding users to specific issues.
  • User-centered design is key: Azimuth was developed with AI practitioners' needs in mind, featuring a dashboard for high-level insights and an exploration space for detailed investigation.

Key Results

  • Azimuth's Case Study (DistilBert on CLINC-OOS):
    • Identified 26 examples for targeted data augmentation using the no_close_train smart tag.
  • Surfaced ambiguity between "restaurant suggestion" and "travel suggestion" intents, and issues with large semantic spaces in intents like "translate" and "define".
  • Found 25 mislabeled examples in the training set and 18 in the validation set using conflicting_neighbors smart tags.
  • Revealed that the model performed worse on short sentences (~5% drop) and examples lacking verbs (~12% drop).
  • Discovered potential annotation artifacts, such as the model's heavy reliance on the word "reservations" for the "accept reservations" intent.
  • The model was found to be underconfident, with a maximum confidence of 90% and an Expected Calibration Error (ECE) of 0.21.
  • Behavioral testing flagged a high failure rate (~22-25%) for typos.
  • Overall Model Performance:
    • 99.2% accuracy on the training set.
  • 93.9% accuracy on the evaluation set without post-processing.
  • Accuracy dropped to 90.7% when a threshold of 0.5 was applied.

Limitations and Future Work

  • Ethical Considerations: The authors acknowledge that Azimuth does not explicitly suggest techniques to detect whether models behave ethically or if datasets contain harmful biases, though some techniques can be adapted for this.
  • Future Work:
    • Expanding Azimuth's capabilities to cover other NLP tasks and domains.
  • Exposing potential ethical concerns in data or ML models.
  • Encouraging community contributions to further refine the tool and process.

Summary updated

BibTeX
@inproceedings{gauthiermelancon2022azimuth,
  author = {Gabrielle Gauthier-melancon and Orlando Marquez Ayala and Lindsay Brin and Chris Tyler and Frederic Branchaud-charron and Joseph Marinier and Karine Grande and Di Le},
  title = {{Azimuth: Systematic Error Analysis for Text Classification}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
}
Figure