Training Set Debugging Using Trusted Items

Xuezhou Zhang, Xiaojin Zhu, Stephen J. Wright

Introduction

Machine learning models are fundamentally limited by the quality of their training data. While large-scale datasets are essential for modern AI, they frequently harbor "bugs"—ranging from simple mislabeled instances to deeply embedded systematic biases. The "garbage in, garbage out" principle remains a significant hurdle; if a historical hiring dataset consistently undervalues specific demographics, a model trained on it will likely perpetuate that bias.

Detecting these errors is difficult. Random outliers are often easy to spot, but systematic biases appear self-consistent to traditional outlier detection algorithms. Manually auditing millions of data points is impossible. However, it is often feasible for human experts to verify a small "trusted" subset of data. This paper introduces DUTI (Debugging Using Trusted Items), an algorithm designed to leverage these few trusted examples to identify and correct thousands of systematic bugs across a massive training set.

Methodology: Debugging as Optimization

DUTI treats the identification of data bugs as a bilevel optimization problem. Instead of simply looking for local inconsistencies, it seeks the most parsimonious explanation for why a model trained on the noisy data fails to predict the labels of the trusted items.

The Core Mechanism

The algorithm searches for the smallest possible set of changes ($\delta$) to the training labels such that a model trained on the modified data perfectly aligns with the trusted items. This is structured as a "problem within a problem":

  1. Outer Level: Minimize the distance between original and corrected labels while ensuring the resulting model satisfies the trusted data.
  2. Inner Level: Perform standard model training (e.g., logistic regression or ridge regression) on the modified training set.
Training data with systematic bias and the bugs identified by DUTI in the Harry Potter toy example.

To make this computationally tractable, the authors relax the discrete label-changing problem into a continuous one. By applying Karush–Kuhn–Tucker (KKT) conditions and the Implicit Function Theorem, they transform the complex bilevel structure into a single-level gradient descent problem. This allows DUTI to produce a ranked list of potential bugs, enabling human experts to review flagged items until their inspection budget is exhausted.

Experimental Evaluation

The authors evaluated DUTI across diverse scenarios, from synthetic toy examples to real-world datasets, including the UCI German Loan and Adult Income datasets to test the detection of historical biases. They also utilized MNIST digits for a complex 10-class task where bugs were simulated using blurred images and neural-network-generated incorrect labels.

Key Results and Performance

DUTI consistently outperformed existing baselines, including Nearest Neighbor (NN), Influence Functions (INF), and Label Noise Detection (LND).

Comparison of Precision-Recall curves for DUTI against baseline methods like Influence Functions (INF) and Nearest Neighbors (NN).

As shown in the Precision-Recall curves, DUTI maintains high precision even as recall increases, indicating that its top-ranked flags are highly likely to be actual bugs. In the MNIST task, DUTI successfully identified over half of the 133 simulated bugs within the first 200 items flagged. Crucially, DUTI does more than just flag errors; it suggests corrected labels. In regression experiments, these suggested fixes almost perfectly reconstructed the underlying "true" function that the model had never seen.

Insights and Future Directions

A primary advantage of DUTI is its "global" perspective. While methods like NN focus on local neighborhood inconsistencies, DUTI understands the logic of the learning algorithm. If a trusted item contradicts an entire cluster of training data, DUTI recognizes that the whole cluster—not just a single point—may represent a systematic bug.

However, the authors note several limitations. A flag from DUTI does not always guarantee a label bug; it could indicate model underfitting, poor extrapolation in data-sparse regions, or inherent noise. Furthermore, the effectiveness of the debugging is highly dependent on the quality and coverage of the trusted items. Future work will focus on scaling the bilevel optimization for "Big Data" contexts and developing strategies for selecting the most informative trusted items to maximize bug discovery.

Summary updated

Abstract

Training set bugs are flaws in the data that adversely affect machine learning. The training set is usually too large for man- ual inspection, but one may have the resources to verify a few trusted items. The set of trusted items may not by itself be adequate for learning, so we propose an algorithm that uses these items to identify bugs in the training set and thus im- proves learning. Specifically, our approach seeks the smallest set of changes to the training set labels such that the model learned from this corrected training set predicts labels of the trusted items correctly. We flag the items whose labels are changed as potential bugs, whose labels can be checked for veracity by human experts. To find the bugs in this way is a challenging combinatorial bilevel optimization problem, but it can be relaxed into a continuous optimization problem. Ex- periments on toy and real data demonstrate that our approach can identify training set bugs effectively and suggest appro- priate changes to the labels. Our algorithm is a step toward trustworthy machine learning.
BibTeX
@inproceedings{zhang2018training,
  author = {Xuezhou Zhang and Xiaojin Zhu and Stephen J. Wright},
  title = {{Training Set Debugging Using Trusted Items}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.1609/aaai.v32i1.11610},
}

Loading related content...

Figure