Auditing Black-Box LLM APIs with a Rank-Based Uniformity Test

Xiaoyuan Zhu, Yaowen Ye, Tianyi Qiu, Hanlin Zhu, Sijun Tan, Ajraf Mannan, Jonathan Michala, Raluca Ada Popa, Willie Neiswanger

TL;DR

The Rank-based Uniformity Test (RUT) is a novel, query-efficient method to audit black-box LLM APIs and verify if a provider is secretly serving a degraded, quantized, or altered model. By comparing a single API response per prompt against a local reference model using a log-rank score, RUT detects subtle model substitutions while remaining completely stealthy to adversarial providers.

What Problem Does This Paper Address?

When you query a large language model (LLM) via a commercial API, you are interacting with a black box. Because serving massive models is incredibly expensive, API providers face a strong financial incentive to secretly swap out the promised model for a cheaper, quantized, or smaller variant. In worse cases, providers might deploy models with hidden fine-tuning or jailbreak-enabling system prompts.

Detecting these covert substitutions is exceptionally difficult. Users don't have access to model weights or full output logits—only the generated text. While previous auditing methods exist, they either require an impractical number of queries or rely on highly specific, artificial prompt distributions. These artificial patterns allow a dishonest provider to easily detect that they are being audited and temporarily reroute those specific queries back to the authentic model to evade detection.

Methods

The authors propose the Rank-based Uniformity Test (RUT), an asymmetric two-sample hypothesis test that operates entirely on natural, user-like queries.

The core workflow is elegant:

  1. Prompting: For a given prompt, the auditor samples exactly one response from the target API and multiple responses from a trusted, locally deployed reference model.
  2. Scoring: Every response is converted into a scalar value using a score function. After evaluating several candidates (including log-likelihood and entropy), the authors found that log-rank—the average of the logarithm of each token's rank under the reference model—offered the highest discriminative power.
  3. Rank Percentile Transformation: The auditor computes the rank percentile of the API's response within the local reference model's score distribution. To handle the discrete nature of text, a randomized quantile residual technique is applied.
  4. Uniformity Testing: Under the null hypothesis (i.e., the API model is identical to the reference model), these rank percentiles must follow a continuous uniform distribution between 0 and 1. Any statistically significant deviation from uniformity, evaluated using the Cramér-von Mises (CvM) test, flags a model substitution.
Table 1: Comparison of LLM auditing methods by accu- racy (Acc.), query-efficiency (Q-Eff.), and robustness to adversarial providers (Rob.).

Datasets

The approach was evaluated across three distinct query domains to ensure generalizability:

  • WildChat: A dataset containing real-world user conversations with ChatGPT, representing typical, unstructured API traffic.
  • BigCodeBench: A specialized benchmark for programming and code generation tasks.
  • MATH: A dataset focused on complex mathematical problem-solving.

Key Insights and Ideas

  • Log-Rank is the Ultimate Signal: Traditional metrics like log-likelihood or predictive entropy struggle to separate subtle model variations. Log-rank amplifies the distributional "idiosyncrasies" of the reference model, making it highly sensitive to even minor shifts in token probabilities.
Distribution of AUROC scores for five candidate score functions
  • Stealth by Design: Because RUT requires only a single API call per prompt and uses natural user queries, the provider cannot distinguish an audit from standard user traffic. This completely neutralizes adversarial "probabilistic routing" or prompt-caching defenses.
  • Resilience to Formatting Noise: Real-world APIs often introduce minor decoding or tokenization discrepancies (e.g., omitting a leading whitespace). String-distance-based methods like Maximum Mean Discrepancy (MMD) fail catastrophically here, flagging false positives, whereas RUT's rank-based approach remains robust.

Key Results

RUT was tested against MMD and a Kolmogorov-Smirnov (KS) baseline across several threat models:

  • Quantization Detection: RUT significantly outperformed baselines in catching 4-bit quantized models. For instance, when auditing a Gemma-2-9b-it model, RUT maintained high statistical power even when the provider only mixed in quantized responses fractionally. However, 8-bit quantization remains a blind spot for all tested methods due to its minimal impact on token distributions.
Statistical power of different methods in detecting substitution of Gemma-2-9b-it with its 4-bit quantized variant
  • Supervised Fine-Tuning (SFT): RUT successfully detected behavioral changes induced by fine-tuning on both benign (Alpaca) and harmful (BeaverTails) datasets within the very first epoch of training.
Figure 3: AUC of SFT checkpoints across epochs.
  • Jailbreaks and System Prompts: In scenarios where providers secretly appended hidden jailbreak prompts (like DAN), RUT achieved near-perfect detection rates (AUCs > 0.85), consistently beating the baselines.

Limitations and Future Work

  • Omniscient Adversaries: The current study does not evaluate an adversary who has full knowledge of the RUT auditing strategy and selectively reroutes prompts specifically engineered to yield typical log-rank statistics.
  • Open-Weight Dependency: RUT requires the auditor to host a local, fully accessible copy of the authentic model to generate reference distributions. Consequently, it cannot be used to audit closed-source, proprietary models (like GPT-4 or Claude) where an authentic local reference does not exist. Relaxing this requirement is a key direction for future research.

Summary updated

Abstract

As API access becomes a primary interface to large language models (LLMs), users often interact with black-box systems that offer little transparency into the deployed model. To reduce costs or maliciously alter model behaviors, API providers may discreetly serve quantized or fine-tuned variants, which can degrade performance and compromise safety. Detecting such substitutions is difficult, as users lack access to model weights and, in most cases, even output logits. To tackle this problem, we propose a rank-based uniformity test that can verify the behavioral equality of a black-box LLM to a locally deployed authentic model. Our method is accurate, query-efficient, and avoids detectable query patterns, making it robust to adversarial providers that reroute or mix responses upon the detection of testing attempts. We evaluate the approach across diverse threat scenarios, including quantization, harmful fine-tuning, jailbreak prompts, and full model substitution, showing that it consistently achieves superior statistical power over prior methods under constrained query budgets.
BibTeX
@inproceedings{zhu2026auditing,
  author = {Xiaoyuan Zhu and Yaowen Ye and Tianyi Qiu and Hanlin Zhu and Sijun Tan and Ajraf Mannan and Jonathan Michala and Raluca Ada Popa and Willie Neiswanger},
  title = {{Auditing Black-Box LLM APIs with a Rank-Based Uniformity Test}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=PSIe9mmF7a},
}
Figure