Distilling Self-Avoidance into Low-Rank Story Adapters

Ask a large language model for a hundred continuations of the same story and many of them will be the same story. Echoes in AI shows this concretely. GPT-4 continued a Kafka passage 100 times. In 50 of those, a policeman gave the same directions, and in 16 a bakery served as a landmark. None recreated Kafka's surreal ending. Human stories scored consistently more original than GPT-4 and LLaMA-3-70B on both WritingPrompts and a set of 2023 TV plot summaries. Models already know what their own typical continuation looks like, because it is simply what they produce most often. They can also be told, in the prompt, to avoid it. Our proposal turns that in-prompt instruction into a small permanent weight change. The model is shown its own most likely continuations and asked to write something unlike them. We keep only the outputs that genuinely differ and remain good, and train a low-rank adapter (LoRA: a small add-on matrix of rank $r$ trained while the base weights stay frozen) to produce such outputs from the bare prompt.

The scientific bet is that "narrative gravity", the pull toward shared tropes, lives in a low-dimensional part of the model. If so, a rank-4 adapter trained on one prompt set should reduce echoing on unseen prompts and genres. It should do so without the cost of long "avoid these" prompts, and without the human preference data that diversity-aware post-training needs. If gravity is instead prompt-specific and high-dimensional, the rank sweep and transfer tests below will show it, and that negative result is itself informative.

Core Claim

Asking the model in context to avoid its own sampled continuations is a self-supervised teacher for diversity. Three things are new. First, the negatives are the model's own continuations sampled at rewind points throughout the story, not only at the opening. Second, the teacher's outputs are filtered for real semantic distance and quality. Third, the result is compressed into a single low-rank adapter that needs no negatives at inference. We claim this adapter reduces plot echoing across the whole length of a story, not just in the first paragraph. We further claim it matches or beats the in-context teacher at a fraction of its inference cost, and that most of the gain survives at very low rank.

Research Program

The teacher builds on two lines of in-context work. In-Context Diversification generates candidate sentences, feeds them back, and asks for different ones. It improved a quality–diversity harmonic mean by over 6% on CommonGen, ComVE and DimonGen, but only for single sentences. The rewind procedure behind the Sui Generis score in Echoes in AI motivates where to place the negatives. That procedure cuts a story every 50 words and samples 20 alternative continuations from each cut. We borrow this structure for training rather than evaluation.

For each WritingPrompts training prompt $x$ and each 50-word cut point, the base model (Llama-3.1-8B-Instruct, the model Chung et al. used) samples $K$ continuations. We call this set of negatives $N$. The same frozen model is then prompted with the story so far, the negatives, and an instruction $c$ to continue in a direction none of them take. Its outputs pass a filter. A candidate $y$ is kept only if its maximum embedding similarity to any element of $N$ falls below a threshold $\tau$, and an LLM judge rates its coherence and fluency above a threshold $q$. The adapter weights $BA$ are then trained on the kept samples:

$$\mathcal{L}(A,B) = -\,\mathbb{E}_{x,\;y \sim \text{filter}(p_{W_0}(\cdot \mid x, N, c))}\,\log p_{W_0 + BA}(y \mid x),$$

Here $W_0$ are the frozen base weights and $B$, $A$ are the rank-$r$ LoRA factors. The student never sees $N$ or $c$, so it must learn to leave its own modal continuation unprompted.

Standard LoRA supplies the method. The claim that LoRA preserves pre-trained calibration (Xu) is a reason to expect it to avoid the quality collapse that full-weight diversity training risks. Chung et al. report that their DDPO and DORPO methods degrade when only four responses per prompt are available. ICM-LoRA is the closest prior work joining in-context signals and LoRA. It trains a conditional VAE on a bank of existing task adapters and generates new adapter weights from averaged hidden states. It was tested on COCO detection and The Pile. We need no adapter bank, and our target is divergence from the model's own mode rather than an invariant task mapping. We deliberately avoid the weight-generator step until a single distilled adapter has been shown to fail.

Provable In-Context Vector Arithmetic constrains the design. It proves that conditioning on raw demonstration pairs encourages copying of surface statistics rather than retrieval of an abstract concept. Showing negatives in context therefore risks a teacher that paraphrases them, which would amplify echoing. The filter is our guard against this. We will report the teacher's raw copy rate as a direct test of this tension in long-form generation, which the literature has not examined.

Evaluation uses three datasets. WritingPrompts serves as the in-domain held-out set. ReedsyPrompts tests genre transfer. Wiki Plot Summaries, which covers 2023 episodes outside the training cutoff, tests out-of-distribution plots. The primary metric is Sui Generis, which correlates 0.55 with human judgments of surprise. It costs roughly 910 LLM calls per 500-word story, so we compute it on 100 stories per condition.

A circularity threat follows from the training setup. The student was trained to avoid its own base model's continuations, so Sui Generis computed against those same continuations would flatter it. We therefore also compute echoes against continuations from a different model family. Echoes in AI shows that cross-model echoes are common, so this is a meaningful test.

Secondary metrics come from Avoidance Decoding and from human judgment. We generate 15 branches per prompt and report cross-branch semantic and style diversity, along with self-BLEU, ROUGE-L and METEOR repetition. We also report per-position Sui Generis curves to capture the "drop ratio" (a model introduces one surprise, then collapses back into tropes). Human pairwise ratings cover quality and surprise.

Baselines

The naive reference is the base model with tuned temperature and nucleus or min-p sampling. Its sampling settings are searched with the same budget used for every other method, and it is reported at matched quality. The model is fixed at Llama-3.1-8B-Instruct for every method, and all methods get the same prompts and a common quality floor. Every diversity number is read only at points where the judge's quality score is within a fixed margin of the base model's. A method beats a baseline only if it has higher cross-model Sui Generis at that matched quality.

The strongest prompting competitor is the in-context teacher itself, run at inference with full negatives. It is an ICD-style long-form extension of In-Context Diversification. No long-form number for it exists in the file. The only reported figure is the over-6% harmonic-mean gain on the short commonsense tasks.

The strongest decoding competitor is Avoidance Decoding. It penalizes similarity to previously generated branches, at the concept level early in generation and at the narrative level later. It reports up to 2.6 times higher output diversity and an average 30% reduction in repetition on WritingPrompts and ReedsyPrompts at 200 tokens. We run it at 200 tokens to reproduce those numbers, then at 500 tokens, and we report wall-clock decoding time.

The strongest training competitor is DDPO from Chung et al.. It adds a semantic and style deviation term to the preference objective and reportedly matches GPT-4o in quality with higher diversity than DPO. The file gives no numeric scalar for it (no reported result found on Lacuna for DDPO's diversity score). We train it as a LoRA of the same rank on the same prompts, so the comparison isolates the training signal rather than parameter count.

Ablations isolate the key ingredient. One trains the same adapter on unfiltered teacher outputs. Another uses a teacher prompted "be original" with no negatives, which removes the self-avoidance signal. A third applies LoRA supervised fine-tuning on human WritingPrompts stories. A fourth replaces the adapter with an activation-steering vector, computed as the mean hidden-state difference between the teacher and plain passes. This follows how Iterative Vectors differences few-shot and zero-shot activations, and how task vectors are patched in.

Why not just…?

Why not just raise the temperature? The Kafka result shows that stochastic sampling from GPT-4 still lands on the same policeman in half of its samples. In-Context Diversification reports that high temperature trades diversity for degeneration. We will not rely on these reports alone. The matched-quality temperature sweep is a first-class baseline, and if it matches the adapter, the adapter adds nothing.

Why not just keep the negatives in the prompt at inference? This is the most serious threat, because the teacher is by construction a strong system. The adapter's advantages would be cost and filtering. With 15 branches of 500 words, the teacher's context grows with every branch, and each rewind point needs fresh negative sampling. The student needs none of this. The filter may also let the student exceed the teacher, which is unfiltered at inference. If the teacher at inference beats the student at matched quality and the cost gap is small, we will report the contribution as only an efficiency result.

Why not just fine-tune on human stories, or run DDPO with LoRA? Chung et al. report that standard post-training homogenizes outputs, which argues against plain supervised fine-tuning on human stories. DDPO is a real competitor. Its needs differ from ours, though: it requires many rated responses per prompt, and it degrades at four. Our signal is self-generated and targets this specific model's own tropes. The same-rank DDPO-LoRA comparison settles which signal is better. If DDPO wins, the self-supervised teacher is not worth its complexity.

Why not just steer activations with a vector? Task vectors recover 80–90% of in-context accuracy only on single-token tasks. Iterative Vectors reports that such vectors are fragile and sometimes worse than plain prompting, again only for classification. Whether a fixed hidden-state offset can steer 500 words of plot is unknown. The steering-vector ablation answers this directly. A rank-1 adapter is close to a vector, so the rank sweep also shows where the two approaches diverge.

Decisive Experiments

The first check is transfer. An adapter trained only on WritingPrompts must raise cross-model Sui Generis on ReedsyPrompts and Wiki Plot Summaries at matched quality. If gains appear only in-domain, the adapter has memorized prompt-specific tropes rather than learned to leave the mode.

The second check is the rank sweep over $r \in \{1, 4, 16, 64\}$. The claim that gravity is low-dimensional predicts that rank 4 retains most of the rank-64 gain. A steady rise with rank refutes it.

The third check is the per-position Sui Generis curve. The base model's originality should decay across 50-word chunks, and the student's should stay flat. The rewind-point negatives exist to produce this flat curve, and the opening-only ablation should show the decay return.

The fourth check is the Kafka probe. We count the policeman and bakery continuations out of 100 for the base model, the teacher and the student. This is a small, legible test of whether the tropes themselves move.

The fifth check reports the teacher's raw copy rate against its negatives, which tests the surface-copying warning from Provable In-Context Vector Arithmetic in long-form text. The project fails if either the matched-cost teacher or the same-rank DDPO-LoRA equals the student on these checks.