Cautious Deep Learning

Yotam Hechtlinger, Barnabás Póczos, Larry A. WassermanarXiv 2018 · OpenReview

TL;DR

This paper introduces "Cautious Deep Learning," a novel approach to classification that addresses the problem of overconfident predictions by outputting sets of possible labels rather than single, definitive ones. By leveraging conformal prediction and focusing on $p(x|y)$ instead of $p(y|x)$, their method is "cautious," returning a "null set" (I don't know) for inputs that don't resemble the training data, and is robust to adversarial attacks.

What Problem Does This Paper Address?

Traditional classifiers, especially deep learning models, often exhibit a "hubristic bias." They are trained to predict a single label by finding the most likely class ($p(y|x)$). However, this can lead to overconfidence, even for inputs that are far from the training data distribution. Imagine a classifier trained on images of cats and dogs; it might confidently label a blurry, abstract image as "cat" or "dog," even though it has no real basis for doing so. This is problematic because it provides a definitive answer when the model is uncertain. The paper aims to create classifiers that are more honest about their uncertainty, providing a measure of confidence and a way to signal when they truly don't know.

Methods

The core of the proposed method is conformal prediction, a framework that provides distribution-free confidence guarantees. Instead of predicting a single label $y$, the model predicts a set of possible labels $C(x)$ such that the true label $Y$ is in $C(X)$ with a guaranteed probability of $1-\alpha$.

What's novel here is their specific construction:

  • Focus on $p(x|y)$: Unlike most classifiers that estimate $p(y|x)$, they estimate the likelihood of observing the features $x$ given a class $y$. This is crucial because it allows the model to assess how well a given input $x$ fits the characteristics of each class $y$, rather than just how likely a class is given the input.
  • Cautious Prediction: By using $p(x|y)$, the classifier becomes "cautious." If an input $x$ is very different from the data seen for any class $y$, the estimated $p(x|y)$ will be low for all classes. This leads to the output of the null set ($\emptyset$), signifying "I don't know."
  • Conformal Prediction Sets: They construct prediction sets $C(x) = \{y : p(x|y) \ge t_y\}$, where $t_y$ is a threshold determined by the desired confidence level $1-\alpha$ and the empirical distribution of $p(x|y)$ on the training data for class $y$.
  • Class Adaptivity: The method learns $p(x|y)$ independently for each class, making it easy to add or remove classes without retraining the entire model.

Datasets

The paper evaluates its method on several large-scale datasets:

  • ImageNet ILSVRC: A large-scale visual recognition dataset with 1,000 classes, used to test the method's performance on a complex, high-dimensional classification task.
  • CelebA: A large-scale face attributes dataset with celebrity images, used for gender classification.
  • IMDB-Wiki: Another large-scale dataset of facial images, also used for gender classification. This dataset is noted for its richer variety of human postures compared to CelebA.

Key Insights and Ideas

  • The "Hubristic Bias" is a Real Problem: The paper clearly articulates how standard classifiers can be overconfident, providing definitive answers even when they have very little evidence.
  • $p(x|y)$ is Key to Cautiousness: Shifting the focus from $p(y|x)$ to $p(x|y)$ is the fundamental insight that enables the "cautious" behavior and the ability to output a null set. It allows the model to say "this doesn't look like anything I've seen before."
  • Robustness to Adversarial Attacks: A significant finding is that adversarial attacks are more likely to be classified as the null set or have the true label included in the prediction set, making the method inherently more robust.
  • Uncertainty Quantification: The method quantifies two types of uncertainty: ambiguity (when the input could belong to multiple classes) and outlyingness (when the input doesn't resemble any class).

Key Results

  • ImageNet: The method achieves competitive performance on ImageNet, with a naive $p(x|y)$ estimation yielding results comparable to established methods, despite learning class densities independently.
  • Gender Recognition: The proposed method shows significantly better generalization on the IMDB-Wiki dataset compared to standard classifiers, which fail to generalize from CelebA. This highlights its robustness to domain shifts.
  • Adversarial Robustness: Table 1 shows that for untargeted attacks, the method maintains reasonable accuracy, and for targeted attacks, it often predicts the null set, indicating robustness.
  • Outlier Detection: Figure 4 demonstrates that random noise inputs are consistently flagged as the null set, even with a lenient confidence threshold.

Limitations and Future Work

  • Computational Cost: Evaluating a large number of kernel density estimators during prediction can be slow compared to standard methods.
  • Decision Boundaries: The independent learning of $p(x|y)$ for each class means decision boundaries might not be as refined as standard methods that consider class interactions.
  • Future Work: The authors suggest exploring more complex estimation functions for $p(x|y)$ to achieve more refined decision boundaries. They also propose addressing the computational cost by developing more efficient ways to learn ordering-consistent approximations of $p(x|y)$ that can be deployed on GPUs. They also mention the need for theoretical work on how to handle a growing number of classes.

Summary updated

Abstract

Most classifiers operate by selecting the maximum of an estimate of the conditional distribution $p(y|x)$ where $x$ stands for the features of the instance to be classified and $y$ denotes its label. This often results in a {\em hubristic bias}: overconfidence in the assignment of a definite label. Usually, the observations are concentrated on a small volume but the classifier provides definite predictions for the entire space. We propose constructing conformal prediction sets which contain a set of labels rather than a single label. These conformal prediction sets contain the true label with probability $1-\alpha$. Our construction is based on $p(x|y)$ rather than $p(y|x)$ which results in a classifier that is very cautious: it outputs the null set --- meaning "I don't know" --- when the object does not resemble the training examples. An important property of our approach is that adversarial attacks are likely to be predicted as the null set or would also include the true label. We demonstrate the performance on the ImageNet ILSVRC dataset and the CelebA and IMDB-Wiki facial datasets using high dimensional features obtained from state of the art convolutional neural networks.
BibTeX
@misc{hechtlinger2018cautious,
  author = {Yotam Hechtlinger and Barnabás Póczos and Larry A. Wasserman},
  title = {{Cautious Deep Learning}},
  year = {2018},
  eprint = {1805.09460},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1805.09460},
}
Figure