TL;DR
Frankentexts introduces a narrative generation paradigm where large language models act as collagists rather than primary authors, assembling new stories out of thousands of randomly sampled human-written text snippets. By constraining the LLM to copy 75%–90% of its output verbatim, the resulting narratives significantly surpass standard autoregressive generation in literary voice and diversity while evading state-of-the-art AI text detectors.
What Problem Does This Paper Address?
Standard autoregressive decoding in LLMs often produces repetitive, clichéd stories plagued by predictable tropes and formulaic phrasing (often dubbed "AI slop"). At the same time, existing AI text detectors rely on a binary assumption—either a passage is machine-generated or human-authored.
The authors investigate whether reasoning LLMs can implicitly navigate the combinatorial search space of selecting, reordering, and minimally editing disparate human-written fragments to produce a coherent narrative. In doing so, the authors examine whether this collage approach can improve literary quality while exposing critical blind spots in AI detection ecosystems.
Methods
The Frankentext pipeline treats text generation as a constrained assembly problem:
- Sampling: The LLM is provided with a prompt and a large context pool (1,500 to 10,000 paragraphs) randomly sampled from human-authored literature.
- Drafting under Constraint: A reasoning model (such as Gemini 2.5 Pro or GPT-5) generates a story under the instruction that a pre-specified proportion (e.g., 90%) must be copied verbatim from the snippets, using only minimal connective phrasing.
- Iterative Polishing: The model executes up to three self-correction passes to resolve narrative inconsistencies, point-of-view shifts, and grammatical clashes introduced by splicing fragments from different books, stopping when the model returns "no edits."
- Agentic Retrieval (Exploration): The authors also experimented with an agentic setting using the Model Context Protocol (MCP) to retrieve relevant snippets via a FAISS semantic index instead of random sampling.
Datasets
- Books3: A dataset of 197K books (>160 million snippets) filtered for paragraph length, English text, and high quality (using the
MBERT-WQRM-Rreward model) to serve as the pool of human-written source material. - Mythos: A dataset of creative writing prompts sourced from Reddit’s
r/WritingPrompts(100-prompt evaluation subset) used to avoid data contamination. - Additional Prompts & Corpora: Tell Me a Story (complex, multi-constraint story prompts), The Count of Monte Cristo (single-author ablation), and the HUMAN DETECTORS news corpus (non-fiction news generation).
Key Insights and Ideas
- Random beats targeted retrieval: Counterintuitively, giving the LLM an agentic tool (MCP) to semantically retrieve relevant paragraphs performed worse than feeding it randomly sampled snippets. Semantic search caused models to become verbose, introducing more of their own text and dropping verbatim copy rates from 75% down to 43%–45%.
- Eradication of AI "fingerprints": Vanilla generations repeatedly fall back on telltale AI words and names (e.g., the name "Elara" appeared 113 times in vanilla samples). In 90% Frankentexts, its occurrence dropped to 2, effectively removing stylistic watermarks.
- Detectors break under mixed authorship: Current detectors are ill-equipped for collaged text. Because the local sentences are genuine human writing, classifiers like FastDetectGPT and Binoculars failed entirely, and Pangram misclassified the majority of stories as human.
- Coherence decay over length: Models follow the copy constraint well in the opening sections, but verbatim copy rates decline by ~10% toward the final sections, leading to increased AI detectability later in the text.
Key Results
- Writing Quality: On a 1–7 Likert scale judged by Claude, Frankentexts consistently outperformed vanilla outputs (Gemini 2.5 Pro rose from 3.18 to 4.21; GPT-5 jumped from 4.20 to 5.88). In human pairwise evaluations, Frankentexts scored higher across all dimensions, with the largest gains in language use (+0.65) and overall interest (+0.53).
- Novelty & Diversity: On NoveltyBench metrics, Gemini Frankentexts achieved a cumulative utility score of 9.27/10 (vs. 6.41 for vanilla) and generated 2.74 distinct semantic clusters across runs (vs. 1.76 for vanilla).
- Adherence & Detection Evasion: Gemini 2.5 Pro and GPT-5 achieved 75% and 82% actual verbatim copy rates, respectively. While 100% of vanilla outputs were detected as AI by Pangram, 72% of Gemini (5k snippets) and 79% of GPT-5 Frankentexts were misclassified as human-written.
Limitations and Future Work
- High Computational Cost: Frankentext generation requires massive context windows and multiple reasoning passes, costing roughly 100–200× more than standard decoding (e.g., $0.81 vs. $0.0085 per Gemini generation).
- Surface Inconsistencies: Human raters noted occasional abrupt tonal shifts (50%) and awkward transitions (40%) across stitched segments.
- Copyright and Ethical Dilemmas: The method relies on copying large verbatim chunks of potentially copyrighted human text, posing severe plagiarism and copyright infringement risks for self-publishing platforms.
- Future Directions: Developing token-level and mixed-authorship attribution detectors, reducing computational overhead through better snippet retrieval, and testing longer-form outputs beyond 500 words.
Summary updated