Just rotate it! Uncertainty estimation in closed-source models via multiple queries

Konstantinos Pitas, Julyan ArbelarXiv 2024 · DOI · arXiv

Introduction

In the modern AI landscape, many state-of-the-art models are deployed as "closed-source" systems accessible only via APIs. While these models, such as GPT-4 Vision or proprietary medical diagnostic tools, offer high performance, they typically provide only a "top-1" prediction (e.g., "This is a giraffe") without a reliable measure of confidence. This lack of transparency is a significant hurdle for high-stakes applications like autonomous driving or healthcare, where understanding when a model is uncertain is as critical as the prediction itself.

Methodology: Test-Time Augmentation for Black-Box Models

The authors propose a simple yet effective "query-only" method to estimate uncertainty using Test-Time Augmentation (TTA). Instead of relying on internal model layers (like the softmax distribution), which are hidden in black-box scenarios, the method queries the model multiple times with slightly modified versions of the input image.

Workflow of the query-only uncertainty estimation method.

The process involves:

  1. Multiple Queries: An image $x$ is transformed into $S$ variations (using rotations, elastic deformations, or Gaussian noise).
  2. Frequency Counting: The system counts how many times the model predicts the original class $A$ out of the $S$ total queries. This fraction, $p_A$, serves as the basis for the uncertainty score.
  3. Natural Transformations: A key contribution of this work is the emphasis on "natural" transformations. While Gaussian noise is a common research tool, the authors find that simple rotations or "wiggles" better mimic the way models process features, leading to more stable results.

Theoretical Foundation: Latent Space Consistency

The researchers provide a mathematical link (Proposition 2.1) between the observed query counts ($p_A$) and the model's internal categorical distribution. By assuming that the model's latent space contains Gaussian noise, they can map the frequency of a specific prediction to a calibrated confidence score using a non-linear function.

Geometric analysis of latent space noise and decision boundaries.

The paper identifies two primary sources of error in uncertainty estimation:

  • Source 1: Noise behaving inconsistently across different images.
  • Source 2: Using an incorrect mathematical model for the noise distribution.

The study demonstrates that natural transformations like rotation produce a more "consistent" noise profile in the model's latent space compared to static Gaussian noise, which often results in "messier" distributions.

Experimental Results

The method was rigorously tested across standard benchmarks, including CIFAR-10, CIFAR-100, and ImageNet, using architectures ranging from ResNet and MobileNet to Vision Transformers (ViT).

  • Calibration Improvements: On CIFAR-100, the approach improved the Expected Calibration Error (ECE) by up to 30% and the AUROC by 40%.
  • Efficiency: Remarkably, the authors found a "two-query miracle" on ImageNet. Querying the model just twice per image was sufficient to significantly improve calibration over the naive baseline of assuming 100% confidence for every top-1 result.
  • Transferability: The "noise profile" can be learned on an open-source model (e.g., a ResNet trained on CIFAR) and then applied to calibrate a completely different closed-source model.

Discussion and Limitations

While the method is highly effective, it introduces an "API tax." Querying a model multiple times increases both latency and financial cost. However, the finding that even two queries provide substantial benefits helps mitigate this concern. Additionally, the current framework primarily addresses binary "Class A vs. Not Class A" scenarios, which may not fully capture the complexity of multi-class uncertainty where a model is split between two highly similar categories.

Future research may extend these input transformation techniques to the latent spaces of Large Language Models (LLMs), potentially providing a universal way to make black-box AI admit when it is unsure.

Summary updated

Abstract

We propose a simple and effective method to estimate the uncertainty of closed-source deep neural network image classification models. Given a base image, our method creates multiple transformed versions and uses them to query the top-1 prediction of the closed-source model. We demonstrate significant improvements in the calibration of uncertainty estimates compared to the naive baseline of assigning 100\% confidence to all predictions. While we initially explore Gaussian perturbations, our empirical findings indicate that natural transformations, such as rotations and elastic deformations, yield even better-calibrated predictions. Furthermore, through empirical results and a straightforward theoretical analysis, we elucidate the reasons behind the superior performance of natural transformations over Gaussian noise. Leveraging these insights, we propose a transfer learning approach that further improves our calibration results.
BibTeX
@misc{pitas2024just,
  author = {Konstantinos Pitas and Julyan Arbel},
  title = {{Just rotate it! Uncertainty estimation in closed-source models via multiple queries}},
  year = {2024},
  eprint = {2405.13864},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2405.13864},
}
Figure