Black-box Detection of LLM-generated Text Using Generalized Jensen-Shannon Divergence
Lists
Only you see this
Abstract
We study black-box detection of machine-generated text under practical constraints: the scoring model (proxy LM) may mismatch the unknown source model, and per-input contrastive generation is costly. We propose SurpMark, a reference-based detector that summarizes a passage by the dynamics of its token surprisals. SurpMark discretizes surprisals into interpretable states, estimates a state-transition matrix for the test text, and scores it via a generalized Jensen–Shannon (GJS) gap between the test transitions and two fixed references (human vs. machine) built once from existing corpora. {\color{blue} Theoretically, we derive design guidance for how the discretization bins should scale with data and provide a principled justification for our test statistic.} Empirically, across multiple datasets, source models, and scenarios, SurpMark consistently matches or surpasses baselines; {\color{blue} our experiments on hyperparameter sensitivity exhibit trends that our theoretical results help to explain, and are consistent with the method’s underlying intuitions.
BibTeX
@inproceedings{chen2026black,
author = {Shuangyi Chen and Ashish J Khisti},
title = {{Black-box Detection of LLM-generated Text Using Generalized Jensen-Shannon Divergence}},
booktitle = {International Conference on Machine Learning},
year = {2026},
url = {https://openreview.net/forum?id=E2kWotHfT7},
}