Reverse Engineering Configurations of Neural Text Generation Models

Yi Tay, Dara Bahri, Che Zheng, Clifford Brunk, Donald Metzler, Andrew Tomkins

Abstract

Recent advancements in neural text generation have raised concerns regarding the provenance of machine-generated content. This study introduces the Machine Configuration Discrimination (MCD) task, demonstrating that neural models leave unique "fingerprints" in their output. By training classifiers to identify specific model settings—such as size, sampling strategy, and input context—we show that these artifacts are detectable and often tied to stylistic and vocabulary choices.

Introduction

While "Deepfake" detection focuses on distinguishing human from machine text, understanding the specific origin of AI-generated content remains a significant challenge. This research investigates whether it is possible to reverse-engineer a model's configuration solely from its output. We explore whether variations in model size (e.g., "Base" vs. "Mega") or sampling parameters (e.g., nucleus sampling thresholds) leave distinct, detectable signatures. Understanding these fine-grained artifacts is crucial for online safety and uncovering the fundamental architectural characteristics of generative models.

Experimental Framework

To evaluate these signatures, we define the Machine Configuration Discrimination (MCD) task. We utilize the Grover model, a transformer-based news generator, to produce text across three primary axes of variation: model size, sampling method, and conditioning context.

Table 1: Definition of MCD tasks across different model parameters, including nucleus sampling (p), top-k sampling (k), conditioning length (l), and model size (S).

The researchers used the CNN/Dailymail news corpus as a seed, generating a large-scale dataset of machine-generated articles labeled with their respective configurations. We then evaluated several classifier architectures, ranging from simple Bag-of-Words (BoW) baselines to complex Transformer encoders, to determine the detectability of these artifacts.

Results and Analysis

Our results indicate that modeling choices are far from invisible. All tested classifiers significantly outperformed random chance, with an average accuracy gain of +66% across the primary MCD tasks.

Table 2: Classification accuracy across various MCD tasks. The results show significant gains over chance, particularly for sampling-related tasks (P and K), with simple BoW models performing competitively against complex architectures.

Key Findings

  • Sampling as a Primary Signal: Sampling methods (Top-$p$ and Top-$k$) proved to be the most easily detectable configurations. For certain nucleus sampling tasks (P3), classifiers achieved gains as high as +199% over the chance baseline.
  • Stylistic Artifacts: Surprisingly, simple Bag-of-Words (BoW) models performed comparably to sophisticated Transformers. This suggests that model "fingerprints" are primarily encoded in word choice and vocabulary distribution rather than complex syntactic or grammatical structures.
  • Magnitude of Difference: It is significantly easier to distinguish between widely different configurations (e.g., $p=0.95$ vs. $p=0.75$) than those with subtle differences, indicating that artifacts follow a predictable magnitude relative to parameter changes.

Conclusion

This work demonstrates that neural text generation models leave ubiquitous and detectable signatures. While distinguishing between different machine configurations is more challenging than simple human-vs-machine detection, it is highly feasible. These findings have significant implications for the development of watermarking techniques and the forensic tracking of AI-generated misinformation. Future work should explore the generalizability of these artifacts across diverse model families beyond Grover to ensure robust provenance tracking in the wild.

Summary updated

Abstract

This paper seeks to develop a deeper understanding of the fundamental properties of neural text generations models. The study of artifacts that emerge in machine generated text as a result of modeling choices is a nascent research area. Previously, the extent and degree to which these artifacts surface in generated text has not been well studied. In the spirit of better understanding generative text models and their artifacts, we propose the new task of distinguishing which of several variants of a given model generated a piece of text, and we conduct an extensive suite of diagnostic tests to observe whether modeling choices (e.g., sampling methods, top-$k$ probabilities, model architectures, etc.) leave detectable artifacts in the text they generate. Our key finding, which is backed by a rigorous set of experiments, is that such artifacts are present and that different modeling choices can be inferred by observing the generated text alone. This suggests that neural text generators may be more sensitive to various modeling choices than previously thought.
BibTeX
@inproceedings{tay2020reverse,
  author = {Yi Tay and Dara Bahri and Che Zheng and Clifford Brunk and Donald Metzler and Andrew Tomkins},
  title = {{Reverse Engineering Configurations of Neural Text Generation Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.25},
}
Figure