Performance Evaluation in Machine Learning: The Good, the Bad, the Ugly, and the Way Forward

Peter A. FlachAAAI 2019 · DOI

In a foundational address at AAAI-19, Peter Flach presented a "state-of-the-union" for machine learning evaluation. Moving beyond the routine reporting of metrics, the paper challenges the community to consider the mathematical validity of how success is measured, asking: What do these numbers actually mean, and are we even allowed to average them?

The Problem: "The Ugly" Sins of Evaluation

Machine learning evaluation is frequently treated as a perfunctory exercise. While models have become increasingly sophisticated, the methods used to judge them often remain mathematically "ill-advised." Flach identifies several critical errors common in current research:

  1. Incoherent Averaging: Researchers often use arithmetic means for metrics on non-linear scales. For instance, the F-score is a harmonic mean; averaging F-scores across cross-validation folds without proper weighting is mathematically incoherent.
  2. Incorrect Interpolation: Linear interpolation is frequently applied to Precision-Recall (PR) curves. However, linear interpolation is only valid in ROC space. In PR space, it leads to meaningless Area Under the Curve (AUPR) values because PR curves require hyperbolic interpolation to remain sound.
  3. Ignoring Scale: Metrics are often treated as simple "ratio scales" (where "twice as much" is a consistent concept) without verifying if the underlying properties support such operations.

Grounding Metrics in Measurement Theory

To address these issues, the paper introduces Representational Measurement Theory to the ML community. This framework ensures that evaluation metrics are grounded in correct mathematical scales (Nominal, Ordinal, Interval, Ratio, or Absolute) and latent properties.

Standard confusion matrix notation used as the basis for deriving performance metrics.

Flach explores concatenation operations—what happens when results, such as confusion matrices from different folds, are combined. For a metric to be "averagable" via an arithmetic mean, it must behave consistently under this concatenation. The paper provides algebraic derivations showing that while Accuracy and True Positive Rate (TPR) correspond to weighted arithmetic means when combined, Precision and F-score do not.

ROC Space, Calibration, and the Psychometric Shift

A key insight of the paper is that calibration is not optional. We should not only care about the decision boundary ($p=0.5$); models must be well-calibrated across the entire probability range to allow for threshold adjustments if class distributions or costs change at test time.

The relationship between ROC space (left), model calibration (middle), and cost-sensitive loss (right).

Flach advocates for viewing ML through the lens of Psychometrics. Just as an SAT score is an indirect measure of a student's latent "ability" on a test of a certain "difficulty," an ML score should be seen as an indirect measure of a model's ability on a dataset of a specific difficulty. This shift suggests using techniques like Beta calibration, which is superior to standard Logistic/Platt calibration for models producing scores on bounded scales.

The Way Forward

The paper concludes by outlining a more rigorous path for ML evaluation:

  • Latent Variable Models: Adopting Item Response Theory (IRT) to grade models more effectively.
  • Causal Inference: Moving toward explaining why an algorithm outperforms another, rather than just reporting that it does.
  • Mathematical Rigor: While implementing proper measurement theories requires more effort than standard library calls, it is essential for the field's scientific integrity.

Summary updated

BibTeX
@inproceedings{flach2019performance,
  author = {Peter A. Flach},
  title = {{Performance Evaluation in Machine Learning: The Good, the Bad, the Ugly, and the Way Forward}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33019808},
}
Figure