Abstract
The growing collaboration between humans and AI models in generative tasks has introduced new challenges in distinguishing between human-written, AI-generated, and human-AI collaborative texts. In this work, we collect a multilingual, multi-domain, multi-generator dataset FAIDSet. We further introduce a fine-grained detection framework FAID to classify text into these three categories, meanwhile identifying the underlying AI model family. Unlike existing binary classifiers, FAID is built to capture both authorship and model-specific characteristics. Our method combines multi-level contrastive learning with multi-task auxiliary classification to learn subtle stylistic cues. By modeling AI families as distinct stylistic entities, FAID offers improved interpretability. We incorporate an adaptation to address distributional shifts without retraining for unseen data. Experimental results demonstrate that FAID outperforms several baseline approaches, particularly enhancing the generalization accuracy on unseen domains and new AI models. It provides a potential solution for improving transparency and accountability in AI-assisted writing.
Introduction
As Large Language Models (LLMs) become ubiquitous, the line between human and machine writing has blurred. We are no longer just in a world of "fully human" or "fully AI" text; we are in the era of human-AI collaboration, where humans polish AI drafts or LLMs extend human ideas.
Existing detection tools struggle with two major issues: granularity and generalization. Most detectors are binary, failing to account for the nuanced "middle ground" of collaborative writing, and they often "overfit" to the specific models they were trained on. When a new model is released or the topic changes (domain shift), accuracy typically plummets. The FAID framework addresses these gaps by shifting the focus from identifying generic "AI-ness" to identifying specific stylistic "fingerprints."
Methods and the FAID Framework
The core innovation of FAID is treating an LLM family (like the Gemini family) as a single author with a consistent stylistic core. This transforms the detection task into a form of authorship attribution.
- Multi-Level Contrastive Learning: Instead of just learning labels, the model learns a high-dimensional embedding space. It is trained to "pull" texts from the same LLM family closer together while "pushing" human text and different LLM families further apart.
- Multi-Task Auxiliary Classification: The system simultaneously learns to classify the text's origin (Human, AI, or Collaborative) while refining these stylistic clusters.
- Inference via Vector Database & Fuzzy kNN: This is the key to generalization. FAID uses a vector database of known styles. When it encounters text from an unseen model, it uses Fuzzy k-Nearest Neighbors (kNN) to find the closest stylistic match in its memory, allowing the system to adapt without retraining.
The FAIDSet Dataset
The researchers introduced FAIDSet, a comprehensive dataset containing 83,350 examples. It is multilingual (English and Vietnamese), multi-domain (academic contexts), and multi-generator (including GPT-4/4o, Llama-3.x, Gemini 2.x, and DeepSeek V3/R1). It specifically labels three types of human-AI interaction: LLM-polished, LLM-continued, and LLM-paraphrased.
Key Results
FAID consistently outperformed state-of-the-art baselines across multiple scenarios. On the FAIDSet test set, it achieved 95.58% accuracy, beating the next best baseline by over 1%.
As shown in the table above, FAID demonstrated remarkable resilience to distributional shifts:
- Unseen Generators: When tested on LLMs it had never seen during training (like Qwen or Mistral), FAID maintained a high accuracy of 93.31%.
- Unseen Domains: In a "stress test" using IELTS essays (a domain not in the training set), FAID achieved 62.78% accuracy, significantly higher than the ~55% achieved by T5-Sentinel.
- Source Identification: FAID identified the correct LLM family with ~79.6% accuracy on its own dataset.
Conclusion and Future Work
FAID provides a robust framework for moving beyond binary detection into the nuanced world of human-AI collaboration. While the current dataset is synthetic by construction, it offers a powerful baseline for identifying stylistic fingerprints. Future directions include expanding the dataset to include low-resource languages and more informal genres, such as social media posts and creative writing, to further test the framework's robustness in real-world scenarios.
Summary updated