SenDetEX: Sentence-Level AI-Generated Text Detection for Human-AI Hybrid Content via Style and Context Fusion

Lei Jiang, Desheng Wu, Xiaolong ZhengEMNLP 2025

TL;DR

This paper introduces SenDetEX, a novel framework for detecting AI-generated sentences within human-AI hybrid texts. It tackles the challenge of short-text detection by fusing stylistic features with contextual information, outperforming existing methods and demonstrating strong robustness.

What Problem Does This Paper Address?

The increasing sophistication of Large Language Models (LLMs) means their generated text is becoming indistinguishable from human writing. This raises concerns about misuse, such as academic misconduct and the spread of misinformation. While AI-generated text detection (AGTD) methods exist, they often struggle with short texts (sentences) and complex human-AI hybrid content where human and AI-generated sentences are mixed irregularly. Existing sentence-level AGTD (S-AGTD) research is limited by a lack of comprehensive evaluation on such hybrid content and fails to leverage crucial contextual information that can help identify the origin of a sentence.

Methods

The core of SenDetEX lies in its "style and context fusion" approach.

  • Feature Encoding: It uses a proxy LLM to extract two types of information from a candidate sentence and its AI-generated counterpart (created using the preceding context as a prompt):
  • Style Features: This includes token probability series (reflecting "precision") and entropy series (reflecting "openness"). AI-generated text tends to be more precise and less open than human text.
  • Contextual Embeddings: Intrinsic semantic embeddings of the candidate sentence and inferred semantic embeddings from the re-generated sentence are captured. AI-generated text often shows a stronger coupling between intrinsic and inferred semantics.
  • Style Extraction Module: This module processes the style features through two parallel branches: a local branch using depthwise separable convolutions for local dependencies and a global branch using a sparsely connected Transformer for global style information. A gating mechanism dynamically fuses these local and global style representations.
  • Information Fusion Module: This module uses a triple cross-attention mechanism to fuse the extracted style features (z_style) with the intrinsic and inferred semantic embeddings (z_ins, z_inf). This allows the model to weigh the importance of different information sources.
  • Classification: The fused features are then fed into a classifier to predict the probability of the sentence being AI-generated.

The authors also introduce AutoFill-Refine, a strategy for synthesizing high-quality human-AI hybrid texts, which is used to construct their benchmark dataset.

Datasets

The paper introduces a new benchmark dataset constructed using the AutoFill-Refine strategy. This dataset is designed to specifically address the limitations of existing S-AGTD research by featuring complex human-AI hybrid content. They also utilize existing corpora like XSUM and WritingPrompts for training and evaluation.

Key Insights and Ideas

  • Context is King (for Sentences): For short texts like sentences, relying solely on intrinsic stylistic features is insufficient. The surrounding context provides crucial clues for authorship attribution.
  • Style + Context = Better Detection: Fusing stylistic signals (precision, openness) with contextual information (intrinsic and inferred semantics) significantly improves sentence-level AI text detection.
  • AI Text has a "Signature": AI-generated text exhibits a distinct pattern in its probability distributions and semantic relationships between original and re-generated versions, which SenDetEX effectively captures.
  • Robustness Matters: SenDetEX demonstrates remarkable robustness against various adversarial attacks (random deletion, substitution, paraphrasing, back-translation), suggesting its learned features are less susceptible to minor perturbations.

Key Results

  • State-of-the-Art Performance: SenDetEX consistently outperforms all baseline models in both in-domain and cross-generator/cross-domain scenarios, achieving significant improvements in F1, AUC, and MCC scores. For instance, in in-domain tests, it improves F1 by 4.7%-5.0% over the best baselines.
  • Strong Transferability: The model shows excellent performance even when tested on data generated by different LLMs or from different domains, highlighting its generalizability.
  • Robustness: Under adversarial attacks, SenDetEX's performance degrades minimally, maintaining high accuracy and outperforming baselines even after attacks. For example, under paraphrasing attacks, it shows improvements of 6.7%-6.8% in F1.

Limitations and Future Work

  • Context Dependency: SenDetEX's effectiveness relies on the availability of preceding context. Its performance might diminish in scenarios with very sparse context, isolated sentences, or fragmented documents.
  • Proxy Model Influence: The performance of the proxy LLM used for feature extraction is crucial. Further exploration into fine-tuning strategies and proxy model selection is suggested.
  • Dataset Optimization: The prompts used for synthesizing the dataset could potentially be further optimized.

The authors plan to address these limitations in future work, focusing on improving performance in low-context scenarios and exploring more advanced fine-tuning techniques.

Summary updated

BibTeX
@inproceedings{jiang2025sendetex,
  author = {Lei Jiang and Desheng Wu and Xiaolong Zheng},
  title = {{SenDetEX: Sentence-Level AI-Generated Text Detection for Human-AI Hybrid Content via Style and Context Fusion}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure