Building Auditable AI Text Detectors with Sparse Autoencoders

Liam Dugan, Callum McDougall, Matthew Denton, Philip Pham, Christine Kaeser-Chen, Chris Callison-BurcharXiv 2026 · arXiv

TL;DR

This paper introduces a novel pipeline to transform "black box" AI text detectors into auditable systems using Sparse Autoencoders (SAEs). By decomposing a neural network's internal activations into thousands of human-readable linguistic features, the authors created a detector that matches state-of-the-art accuracy while providing clear, natural-language rationales for every decision.

What Problem Does This Paper Address?

As AI-generated text becomes ubiquitous, detectors are being deployed in high-stakes environments like academic peer review, admissions, and journalism. However, we face a "transparency gap":

  1. Neural Detectors are highly accurate but operate as "black boxes," making it impossible for a human auditor to understand or contest a "flagged" result.
  2. Stylometric Detectors (which look at sentence length, word frequency, etc.) are interpretable by design but perform significantly worse than neural models.

The authors aim to bridge this gap, creating a system that is both as performant as a fine-tuned transformer and as transparent as a checklist of linguistic "tells."

Methods

The researchers developed a four-stage pipeline to extract "concepts" from a neural detector:

  1. Fine-tuning: They start with a standard transformer (Gemma-2-2B) fine-tuned for AI detection.
  2. SAE Training: They train Sparse Autoencoders on the model’s residual stream. SAEs are designed to "untangle" the complex, overlapping activations of a neural network into a high-dimensional space of sparse, "monosemantic" features.
  3. Auto-Interpretation: An LLM (Gemini-3.1-flash-lite) analyzes the documents that activate each SAE feature and generates a short natural-language description (e.g., "activates on the word 'thorough'").
  4. Fidelity Filtering & Classification: They calculate a "simulation score" to ensure the description actually matches the feature's behavior. Finally, they train a simple, sparse logistic regression model using only these high-fidelity, interpretable features.
The pipeline converts a black box detector into an auditable one by extracting and labeling internal features.

Datasets

The study utilized three primary data sources:

  • COLING Shared Task 1: An "open domain" dataset featuring text from 41 different language models paired with human text.
  • Peer Review Detection Benchmark (PRDB): A "closed domain" dataset specifically for academic peer reviews (Human vs. GPT-4o and Llama-3).
  • RAID: A robust benchmark used for out-of-distribution (OOD) evaluation to see how well the detectors generalize to unseen models and domains.

Key Insights and Ideas

  • LLMs "Know" AI Text: Surprisingly, even SAEs trained on base (non-fine-tuned) models were effective at detection. This suggests that during pre-training, LLMs naturally learn features that distinguish machine-generated prose from human writing.
  • Specific Linguistic "Tells": The system identified highly predictive features that are often invisible to humans but common in AI, such as specific uses of the word "potential," "delve into," or unique markdown formatting like double colons after bold headers.
  • Auditable Decisions: Because the final classifier is a simple linear model over labeled features, the system can provide a "Top Features" list for any given document, showing exactly which phrases pushed the decision toward "AI" or "Human."
Figure 1: In our work, we use Sparse Autoencoders to convert existing black box AI detectors into ones that operate on human-interpretable concepts—allowing for more auditable deployment.

Key Results

  • Performance Parity: The interpretable SAE-based detector matched the AUC and F1 scores of the original "black box" neural detector across multiple benchmarks.
  • Superior Generalization: On out-of-distribution data (the RAID benchmark), the SAE-based models actually outperformed standard neural baselines, suggesting that focusing on specific linguistic features helps prevent the model from over-fitting to training-set artifacts.
  • Causal Validity: Through "paraphrase ablation" experiments, the authors proved that removing the specific words described by their SAE features (e.g., asking an LLM to rewrite a text to remove the word "thorough") successfully flipped the detector's prediction, confirming the features are truly driving the model's logic.

Limitations and Future Work

  • Recall vs. Precision: The authors noted that auto-interpretation pipelines often produce descriptions with high recall but lower precision—they might identify a word but miss the specific context required for the feature to fire.
  • Base Model Dependency: The current work is built on Gemma-2-2B. While SAEs exist for other models, the performance of this pipeline on larger or different architectures remains to be tested.
  • Adversarial Hardening: Now that we can "see" the features detectors use, a logical next step is investigating how to make these features more robust against "humanization" attacks that specifically aim to scrub these linguistic tells.

Summary updated

Abstract

AI-generated text detectors are increasingly deployed in consequential settings---peer review, education, admissions, journalism, and content moderation---where their outputs directly affect individuals. Yet the strongest detectors are fine-tuned neural classifiers whose decisions are opaque, while transparent alternatives (stylometric and linguistic-feature classifiers) underperform them by wide margins. We present a pipeline that converts a fine-tuned neural detector into a classifier that operates on human-readable concepts. We accomplish this by training sparse autoencoders (SAEs) directly on a detector's residual stream, assigning auto-interpretation labels to each SAE feature, and retaining only features whose generated descriptions pass a global fidelity threshold. We then train a lightweight classifier on just the interpretable features. We find that our SAEs recover latent concepts that correspond to highly specific and predictive linguistic "tells" of AI-generated text. Classifiers built on these extracted features are not only highly interpretable but exhibit stronger performance on out-of-distribution texts than either neural or stylometric alternatives.
BibTeX
@misc{dugan2026building,
  author = {Liam Dugan and Callum McDougall and Matthew Denton and Philip Pham and Christine Kaeser-Chen and Chris Callison-Burch},
  title = {{Building Auditable AI Text Detectors with Sparse Autoencoders}},
  year = {2026},
  eprint = {2610.30006},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.30006},
}
Figure