MoSEs: Uncertainty-Aware AI-Generated Text Detection via Mixture of Stylistics Experts with Conditional Thresholds

Junxi Wu, Jinpeng Wang, Zheng Liu, Bin Chen, Dongjian Hu, Hao Wu, Shu-Tao Xia

Abstract

Reliable detection of AI-generated text remains a challenge due to the stylistic diversity of human writing across different domains. Traditional detectors typically apply a static threshold to a discrimination score, leading to significant errors when encountering out-of-distribution styles. We propose MoSEs (Mixture of Stylistics Experts), a novel framework that replaces static thresholds with dynamic, conditional ones. By leveraging a Stylistics-Aware Router and a Conditional Threshold Estimator, MoSEs adapts to the linguistic nuances of the input text. Our evaluation across eight diverse datasets demonstrates an average performance boost of 11.34%, with improvements reaching 39.15% in data-scarce environments.

Introduction

The rapid proliferation of Large Language Models (LLMs) has necessitated robust detection mechanisms. Most existing solutions produce a "discrimination score" and apply a fixed threshold (e.g., 0.5) to classify text as human or machine-written. However, human writing styles—or habitus—vary significantly by domain; the concise style of a news report differs fundamentally from the technical depth of a medical paper.

As illustrated in Figure 2, the distribution of discrimination scores for human versus machine text shifts markedly across datasets like CNN and PubMedQA. This shift confirms that a "one-size-fits-all" threshold is inherently insufficient for accurate cross-domain detection.

Distribution of discrimination scores across different styles, showing the need for adaptive thresholds.

Methodology: The MoSEs Framework

The MoSEs framework addresses the limitations of static thresholds through three core components:

  1. Stylistics Reference Repository (SRR): A comprehensive library containing human and AI-generated texts annotated with linguistic features (e.g., n-gram repetition, type-token ratios) and semantic embeddings.
  2. Stylistics-Aware Router (SAR): Upon receiving a new text, the SAR employs prototype-based clustering to identify stylistically similar samples within the SRR, thereby "activating" the most relevant stylistic experts.
  3. Conditional Threshold Estimator (CTE): The CTE models the input's linguistic and semantic features to calculate a custom, optimal threshold. Two variants were tested: MoSEs-lr (Logistic Regression) and MoSEs-xg (XGBoost).

The complete system architecture is depicted in Figure 1.

Overview of the MoSEs framework, from reference repository to dynamic threshold estimation.

Experimental Results

MoSEs was evaluated on eight datasets, including high-resource (e.g., XSum, SciXGen) and low-resource (e.g., DialogSum, PubMedQA) scenarios.

Performance Gains

MoSEs-xg consistently outperformed baseline methods across all tested score models (RoBERTa, Fast-DetectGPT, and Lastde). On average, the framework improved detection performance by 11.34%.

Robustness in Low-Resource Settings

A key strength of MoSEs is its efficiency in "cold start" scenarios. With only 200 reference samples, MoSEs-xg outperformed standard methods by nearly 40%, indicating a superior ability to learn stylistic boundaries from minimal data.

Stylistic Insights

The inclusion of conditional features, such as text length and log-probability variance, effectively reduces the overlap between human and machine score distributions. These linguistic cues allow the model to identify machine-generated text more reliably by conditioning the decision on the specific style of the input.

Conclusion

MoSEs represents a significant step toward uncertainty-aware AI detection. By moving from static to conditional thresholds, the framework achieves state-of-the-art performance and demonstrates remarkable robustness across domains and model types. Future work will explore extending the framework to multi-class classification to identify specific source LLMs and optimizing inference latency for real-time applications.

Summary updated

Abstract

The rapid advancement of large language models has intensified public concerns about the potential misuse. Therefore, it is important to build trustworthy AI-generated text detection systems. Existing methods neglect stylistic modeling and mostly rely on static thresholds, which greatly limits the detection performance. In this paper, we propose the Mixture of Stylistic Experts (MoSEs) framework that enables stylistics-aware uncertainty quantification through conditional threshold estimation. MoSEs contain three core components, namely, the Stylistics Reference Repository (SRR), the Stylistics-Aware Router (SAR), and the Conditional Threshold Estimator (CTE). For input text, SRR can activate the appropriate reference data in SRR and provide them to CTE. Subsequently, CTE jointly models the linguistic statistical properties and semantic features to dynamically determine the optimal threshold. With a discrimination score, MoSEs yields prediction labels with the corresponding confidence level. Our framework achieves an average improvement 11.34% in detection performance compared to baselines. More inspiringly, MoSEs shows a more evident improvement 39.15% in the low-resource case. Our code is available at https://github.com/creator-xi/MoSEs.
BibTeX
@inproceedings{wu2025moses,
  author = {Junxi Wu and Jinpeng Wang and Zheng Liu and Bin Chen and Dongjian Hu and Hao Wu and Shu-Tao Xia},
  title = {{MoSEs: Uncertainty-Aware AI-Generated Text Detection via Mixture of Stylistics Experts with Conditional Thresholds}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=MckyEXSo9q},
}
Figure