Copying an Author's Word Choices at the Moments a Language Model Would Pick the Cliché

Large language models write in a recognizable average voice. Their sentences fall back on common grammatical patterns, their vocabulary is narrower than a human writer's, and their plots repeat each other. The cause is simple: a model trained to predict the likeliest next word picks the likeliest option wherever several options would do. This proposal attacks the problem at those moments. We add a copy mechanism at decoding time that lets the model take a word straight from a small sample of a target author's own writing instead of from its own vocabulary. Unlike existing copy methods, it copies only at "choice points": positions where the model is torn between interchangeable options, which is exactly where homogenization happens. It does not copy where the next token carries content such as a name or a fact, which is where copying from an unrelated author would leak their subject matter.

The copy mechanism is a pointer-generator, a decoder that blends two distributions at every step, one over its normal vocabulary and one that "points" at tokens in a reference text, using a learned or computed switch. Pointer-generators have been used to keep summaries faithful to their sources. Recent work shows that an ordinary pretrained model can be run as one with no training. Nobody has asked whether the switch can be aimed at style instead of facts. If it can, we get author-specific voice without fine-tuning, without the cost of stitching paragraphs together, and with a single interpretable signal for where the model's voice departs from the author's.

Core Claim

We claim that a training-free pointer over a retrieved author sample will raise style fidelity and grammatical diversity above prompting with the same sample, without leaking the sample's content. The condition is that the copy switch be gated on two things: how spread out the model's probability is across near-equivalent candidates, and how little content the position carries. A fixed blend of copy and generate probabilities should not achieve this.

Concretely, at step $t$ the next-token distribution is

$$P(w) = (1-g_t)\,P_{vocab}(w) + g_t \sum_{i:\,b_i = w} a_{t,i}$$

The terms are defined as follows. $P_{vocab}$ is the model's usual output distribution. $b_i$ is the $i$-th token of the author sample placed in context. $a_{t,i}$ is the model's own attention from the current position to that token, renormalized over the sample. $g_t \in [0,1]$ is the copy switch. The new ingredient is $g_t$, which we define as a product of two terms. The first is a choice-point score: the probability mass the model spreads across top candidates whose embeddings cluster together. High values mean the model is choosing among near-synonyms or equivalent constructions. The second is a content suppressor: one minus the layer-divergence signal that LargePiG uses to detect fact-bearing tokens. LargePiG computes the Jensen–Shannon divergence between early-layer and final-layer predictions and copies more when that divergence is high, because entities keep changing until the last layer. We reuse its signal but invert its role. Copying is turned off at fact-like positions and on at choice points. Named-entity tokens in the author sample are also masked out of the copy distribution, following the entity-masking idea in StyleTunedLM.

How This Relates to Prior Work

The motivating evidence comes from four studies, each measuring the average voice at a different level. Comparing LLM-generated and human-authored news text using formal syntactic theory parses text with a formal English grammar. It finds that every model tested has lower construction entropy than human news writing (H = 3.221–3.284 versus 3.342) and underuses stylistic constructions such as absolute phrases. Quantifying the Risks of LLM- and Tool-assisted Rephrasing to Linguistic Diversity reports vocabulary shrinkage of up to 29% when models rephrase human text. Echoes in AI finds that models echo one another's plot choices. Can AI writing be salvaged? identifies the "rewriting bottleneck": an LLM that correctly flags a cliché tends to replace it with another. Together these studies say the failure is local and repeated: at each substitutable slot, the model picks the mode. That is why we act slot by slot rather than on whole documents.

The method comes from the pointer-generator line. Get To The Point introduced the mixture above with a learned switch for summarization. Dr. Summarize pushed the switch toward copying to preserve clinical terms. LargePiG made the mechanism training-free inside decoder-only models, at about 6% extra latency on LLaMA-2-7B. All three point into a source whose content the output must reproduce. We point into a text whose content the output must not reproduce, and that reversal is why the switch has to be redesigned.

Two results constrain the design. First, Pointer-Generator Networks for Low-Resource Machine Translation: Don't Copy That! shows that unconstrained copying produces degenerate output when the source should not be reproduced. Our setting is closer to theirs than to summarization, so an ungated or weakly gated pointer should fail in the same way. Second, Get To The Point notes the "extractive-heavy" stitching of copied spans. We expect this to appear as copied multi-word runs and will measure it directly.

The closest work is Frankentext. It forces a model, through prompting and agentic retrieval, to compose stories that are up to 90% verbatim human snippets. It raises LLM-judged quality from 3.18 to 4.21 and cluster diversity from 1.76 to 2.74, at 100–200 times the cost of normal decoding and with tonal breaks at snippet boundaries. Frankentext shows that human material breaks the average voice. We test whether a few well-placed tokens, rather than whole paragraphs, can do the same at near-normal cost.

LaMP shows that retrieving a user's past writing into the prompt helps personalization, but it decodes normally. StyleTunedLM instead trains a separate low-rank adapter (LoRA) per author. We are training-free and keep the author's text outside the weights.

Research Program

The system has three parts. The first is retrieval. For each prompt, we retrieve a few thousand tokens of the target author's text by stylistic rather than topical similarity. We use a sentence-level style embedding computed after entity masking, so the retrieved passages are not about the prompt's subject. The second part is the gated pointer defined above. It is run inside a LLaMA-2-7B or Qwen1.5 model, matching LargePiG's backbones so we can reuse its layer-divergence computation. The third part is a small set of calibration choices, made once on held-out authors and then frozen. These include which early layer to compare against, how many candidates define a choice point, and an overall scale on $g_t$.

The analysis half matters as much as the system. We will log $g_t$ at every step and relate the positions where it fires to the grammar parser's construction labels. The question is whether copying concentrates at the modifier, adverbial, and phrase-opening slots that the syntactic study found models underuse, or whether it scatters across arbitrary tokens. If it concentrates, the switch becomes a cheap, per-token diagnostic of homogenization. That matters because Echoes in AI needs roughly 910 LLM calls to score one 500-word story.

Baselines

The naive reference point is the same base model decoding with its default sampling settings, given the same prompt and no author text. It shows the size of the homogenization gap on our prompts.

The most important baseline is few-shot prompting with the identical retrieved author sample in context and the pointer switched off ($g_t = 0$). It uses exactly the same tokens, context length, and compute as our method, so any difference comes from the decoding rule alone. It is the natural skeptic's alternative. StyleTunedLM reports about 69% attribution accuracy for few-shot prompting on its 10-author Gutenberg task. We will rerun this baseline ourselves on our splits rather than rely on that number.

The strongest existing method on the style question is StyleTunedLM, which trains a LoRA adapter per author with entity masking and merges it with an instruction-tuning adapter. It reports 87.9% accuracy with a 10-class BERT author classifier on the Project Gutenberg 10-author dataset. We will retrain it on the same per-author training text we use for retrieval, with the same backbone and a comparable tuning budget, measured in GPU-hours. We will then evaluate both systems on the same classifier and prompts. Matching its attribution accuracy without any training would count as a practical win. Beating it on grammatical diversity or entity leakage at equal attribution would count as a scientific win.

We also run a cost-reduced Frankentext that uses the same retrieved sample as its snippet pool, with its drafting-and-polishing loop. It shows the quality and diversity reachable by macro-stitching and what that costs relative to our roughly single-pass decoding.

Two ablations isolate the key ingredient. The first uses a fixed-blend pointer, a constant $g$ tuned over a grid on held-out authors. This is essentially nearest-neighbor interpolation into an author datastore. The second is a pointer gated by LargePiG's original rule, which copies at fact-like positions. We further ablate each factor of our switch (choice-point score alone, content suppressor alone) and the entity mask. Our method counts as succeeding only if it beats the tuned fixed blend on the joint criterion described below. It must not simply trade leakage for style.

Why not just…?

Why not just put the author's text in the prompt? This is the pointer-off baseline, and LargePiG's own finding that models already behave partly like pointer-generators means the prompt alone may carry some of the effect. The evidence against it being sufficient is indirect. StyleTunedLM's gap between roughly 69% and 87.9% suggests that exemplars in context shift style less than adaptation does. Research Borderlands finds that prompting LLMs to adopt a target field's writing norms changes surface jargon but not underlying structure. Our experiment settles the question directly: the same tokens are in context in both conditions. If gated copying does not add attribution accuracy and construction entropy over pointer-off prompting, the idea fails.

Why not just raise the temperature or use a wider sampling cutoff? Sampling from the tail makes every token more random, including the ones that hold the sentence together. Echoes in AI reports that tropes persist across prompting choices. Nothing in these knobs points toward a particular author. Still, higher temperature may match us on generic diversity metrics such as vocabulary size. We therefore compare along the whole temperature sweep, looking for the diversity each method reaches at matched fluency and judged quality. We also report author attribution, which temperature cannot improve by construction.

Why not just use a fixed-weight blend with the author's text, as in nearest-neighbor language models? This is the real threat, because it is the same machinery without our switch. Our prediction, grounded in the "Don't Copy That!" result, is that any constant weight large enough to shift style will also copy content and create stitched runs. Any weight small enough to avoid that will do little. The fixed-$g$ ablation, tuned on the same held-out authors, tests this. If its best setting matches ours on the joint criterion, we will report that the choice-point switch adds nothing and reposition the contribution as the retrieval-plus-pointer system and its diagnostics.

Why not just fine-tune an adapter per author? Adapters work, as StyleTunedLM's 87.9% shows. But they need a training run per author, they put the author's text into the weights (a privacy concern that LaMP raises), and they are known to leak entities. We do not expect to beat adapters on attribution. The comparison asks whether we get close for free while leaking less.

Proposed Experiments

The primary testbed is the Project Gutenberg 10-author dataset from StyleTunedLM. Each author's text is split into a retrieval pool and a held-out set. The backbone generates 500-word continuations for WritingPrompts prompts that have no connection to the authors' subject matter.

We measure five outcomes. Style fidelity is 10-class author attribution, scored on entity-masked outputs so that copied names cannot inflate it. Grammatical diversity is Shannon entropy over English Resource Grammar constructions, following the syntactic-theory study, and compared with its human reference of 3.342. Lexical diversity is vocabulary size and Jaccard overlap. Leakage has two parts: the rate at which named entities and content nouns from the author's pool appear in outputs, and the share of copied runs longer than four tokens as a stitching measure. Quality is scored by an LLM judge on the 7-point Likert protocol that Frankentext uses, alongside a human pairwise preference study on a subset.

The decisive comparison is on a Pareto frontier of attribution against leakage, traced by sweeping the switch scale for our method and the blend weight for the fixed-blend ablation. The gated pointer must lie strictly above the fixed blend. Sui Generis scores (Echoes in AI) will be computed on 50 stories per condition because of their cost. We will also check whether the rate at which the switch fires correlates with those scores across stories, which is the test of the cheap-diagnostic claim.

A second regime tests transfer to rephrasing, using the 819-text multi-domain corpus from the rephrasing-risk study. Here the pointer targets the text being rephrased itself, with the same switch. The goal is to recover vocabulary overlap toward Grammarly's reported 0.90 Jaccard rather than the below-0.6 range reported for LLM tools. Two guards keep this honest. A judge must confirm that the requested rephrasing actually happened, and we report copy-everything as a trivial upper bound on overlap.

Latency is measured against LargePiG's reported overhead of about 6% on LLaMA-2-7B. The claim of practical cost is falsified if our overhead exceeds a small multiple of that, and it is far below Frankentext's 100–200 times regardless.

Finally, three results would each refute the core claim. If the tuned fixed blend reaches the same frontier, the switch adds nothing. If pointer-off prompting with the same sample matches our construction entropy, the pointer itself adds nothing. If the switch does not fire more often at the underused grammatical constructions than at random slots, it does not track the homogenization mechanism. We treat any of the three as a negative result and will report it as such.