CrossCheck: Rapid, Reproducible, and Interpretable Model Evaluation

Dustin Arendt, Zhuanyi Shaw, Prasha Shrestha, Ellyn Ayton, Maria Glenski, Svitlana VolkovaNAACL 2021

TL;DR

This paper introduces CrossCheck, an interactive Jupyter widget designed to facilitate rapid, reproducible, and interpretable evaluation of machine learning models, particularly for NLP tasks. By moving beyond simple aggregate metrics, CrossCheck allows users to deeply explore model errors, compare multiple models side-by-side, and gain actionable insights into model behavior and limitations.

What Problem Does This Paper Address?

Evaluating machine learning models, especially in complex domains like Natural Language Processing (NLP), goes far beyond just looking at overall performance metrics like accuracy or F1-score. While these metrics give a general idea of performance, they fail to capture the nuances of why a model succeeds or fails. This lack of deeper understanding makes it difficult to:

  • Build trust: Users need to know if a model is reliable and predictable.
  • Improve models: Identifying specific failure modes is crucial for targeted improvements.
  • Understand generalizability: How well does a model perform across different datasets or domains?
  • Compare models effectively: Simply looking at F1 scores between two models doesn't reveal how they differ in their predictions or errors.

Existing tools often focus on interpreting individual model decisions or analyzing errors for a single model. However, there's a need for a tool that supports rapid, iterative comparison of multiple models and allows for a more comprehensive exploration of their strengths, weaknesses, and error patterns in a reproducible manner.

Methods

CrossCheck is implemented as a Jupyter widget, designed for seamless integration into existing data science workflows. Its core innovation is the "histogram heatmap". This visualization extends the traditional confusion matrix by replacing each cell with a histogram. This histogram displays the distribution of a third variable (e.g., question type, confidence score) conditioned on the values of the two variables defining the cell (e.g., predicted vs. actual label).

Key features of CrossCheck include:

  • Interactive Exploration: Users can filter data by clicking on bars in the histograms, allowing for focused analysis of specific subsets of instances.
  • Cross-Model Comparison: The tool is explicitly designed to compare multiple models simultaneously, enabling users to see where they agree and disagree.
  • Reproducibility: By being a Jupyter widget, the entire evaluation process, including the code and visualizations, can be saved and rerun.
  • Normalization Options: Users can choose how histograms are normalized (by table, column, or cell) to better highlight different aspects of the data distribution.
  • Instance-Level Detail: Users can click on histogram bars to view the raw instance data, facilitating in-depth analysis of specific examples.
  • Annotation: The tool allows users to add notes to specific instances, aiding in the documentation of findings.

The development followed a user-centered, iterative process, starting with basic heatmaps and evolving to the final histogram heatmap design based on user feedback.

Datasets

The paper demonstrates CrossCheck's utility across several NLP tasks using established datasets:

  • Named Entity Recognition (NER):
    • CoNLL (Sang, 2003)
    • ENES (Aguilar et al., 2018)
    • WNUT 17 Emerging Entities (Derczynski et al., 2017)
  • Reading Comprehension (RC):
    • Stanford Question Answering Dataset (SQuAD) (Rajpurkar et al., 2016)
    • TriviaQA (mentioned as evaluated, but SQuAD results are primarily presented)
  • Clickbait Detection (CB):
    • Clickbait Challenge 2017 dataset (Potthast et al., 2018)

No new benchmarks were introduced; the focus was on applying CrossCheck to existing tasks and datasets.

Key Insights and Ideas

  • Beyond Aggregate Metrics: The paper strongly advocates for moving beyond simple F1 scores. CrossCheck demonstrates that understanding how models fail (e.g., specific error types, confidence levels for correct/incorrect predictions) is as important as knowing if they fail.
  • The Power of the Histogram Heatmap: This novel visualization effectively bridges the gap between high-level confusion matrices and low-level instance inspection. It allows for rapid identification of patterns across multiple dimensions simultaneously.
  • Reproducibility is Key: By embedding CrossCheck in Jupyter, the entire evaluation process becomes a shareable and reproducible artifact, crucial for scientific rigor.
  • Model Generalizability Insights: The NER use case highlights how CrossCheck can reveal how models trained on different datasets generalize to unseen data, showing unexpected misclassifications (e.g., social media models misclassifying ORG on news data).
  • Confidence-Correctness Relationship: In Reading Comprehension, CrossCheck clearly visualizes that correct predictions generally have higher confidence, but also reveals variability, especially for "why" questions, suggesting areas for model improvement.
  • Identifying Model Disagreements: For clickbait detection, CrossCheck effectively highlights inconsistencies between different models, even those from the same competition, pointing to potential issues in their prediction ranges or strategies.

Key Results

The paper doesn't present traditional quantitative results like "model X achieved Y% accuracy." Instead, the results are qualitative and observational, derived from applying CrossCheck to the use cases:

  • NER: CrossCheck revealed that models trained on social media data misclassify ORG on news data, and news models overpredict named entities on social media data. It also showed how models trained on different datasets generalize to the same test data and how models trained on the same data transfer to different test datasets.
  • Reading Comprehension: CrossCheck demonstrated that models have higher confidence when correct and lower confidence when incorrect, with notable variability in confidence for "why" questions.
  • Clickbait Detection: CrossCheck highlighted inconsistencies in prediction ranges between different models (e.g., "blobfish" and "striped-bass" teams), explaining disparities in F1 scores and showing how to compare coarse vs. fine-grained prediction strategies.

Limitations and Future Work

  • Customization for New Tasks: A limitation is that adding new use cases or data types might require users to write custom JavaScript code to visualize instances in the details sidebar.
  • Future Work: The authors suggest expanding CrossCheck to include additional generic components to support more core NLP or ML tasks. They also aim to make it easier for users to add custom visualizations for new data types.

Summary updated

BibTeX
@inproceedings{arendt2021crosscheck,
  author = {Dustin Arendt and Zhuanyi Shaw and Prasha Shrestha and Ellyn Ayton and Maria Glenski and Svitlana Volkova},
  title = {{CrossCheck: Rapid, Reproducible, and Interpretable Model Evaluation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure