TL;DR
LLM-generated fiction is notoriously repetitive, often over-relying on a handful of "AI slop" names like Elara and Elias. This paper introduces speculative naming, a fine-tuning-free, API-compatible method that reduces name repetition by 95% while maintaining the naturalness of character names.
What Problem Does This Paper Address?
If you’ve spent enough time reading AI-generated stories, you’ve likely met a lot of characters named Elara, Elias, or Leo. This phenomenon, dubbed "name slop," is a visible symptom of mode collapse—a narrowing of the model's creative output often caused by alignment processes like RLHF and DPO.
While researchers have developed ways to fix this at the "logit" level (the raw probability scores of the model), most users only have access to models via APIs. Simple fixes like "don't use these names" in a prompt often fail due to the balloon effect: if you ban Elara, the model simply picks a new favorite name and repeats it just as frequently. This paper seeks a way to control name concentration at the corpus level without needing access to model weights or retraining.
Methods
The authors propose Speculative Naming, which adapts the "draft-verify-reject" paradigm of speculative decoding to the level of character names. The process involves two passes:
- Scaffold: The model generates a story using placeholders like
[CHAR_1]. - Draft: The model is asked to provide several candidate sets of names with "verbalized probabilities" (asking the model to state how likely it thinks a name is).
- Verify: Each name is scored using a SlopTracker. This tracker combines a "baseline fingerprint" (names the model is known to over-use) with a "runtime counter" (names used in the current generation run).
- Select: A selection rule (either a soft Boltzmann reweighting or a "HARD" threshold) chooses the best name set.
- Substitute: The placeholders in the original story are replaced with the chosen names.
The core logic is captured in Algorithm 1, which inserts this speculative layer between the two passes of a standard story generation pipeline.
To measure success, the authors introduce the Slop Ratio (SR), a list-free metric based on Simpson’s concentration index that calculates the probability of two characters in a corpus sharing the same name.
Datasets
The researchers conducted a massive empirical study to validate their method:
- Corpus: 44,782 stories generated across 20 different LLMs (including Gemma, Qwen, GPT-4, and Llama families).
- Languages: Stories were generated in English, Spanish, French, and German.
- Stimuli: Models were prompted with either text-based or visual (pictogram) stimuli to ensure variety in story structure.
- Validation: 88,919 named characters were extracted and matched against civil-registry name lists (like US Social Security data) to check for "naturalness."
Key Insights and Ideas
- The Balloon Effect: Static banlists are ineffective. When the top 20 slop names are banned, models simply displace their concentration onto new "emergent" slop names. For example, banning Tiber in Gemma led to a surge in characters named Hemlock.
- Language-Specific Slop: While some names like Leo are universal attractors, slop is largely language-specific. A banlist for English won't help a model writing in German.
- Naturalness vs. Novelty: Banlists often force models to invent "fantasy-style" names that feel unnatural. Speculative naming, by contrast, keeps the name distribution close to human-authored baselines.
- Vision Helps: Interestingly, providing a visual stimulus (images of pictograms) reduced name slop in most models compared to text-only prompts, likely because the visual grounding forced more specific creative choices.
Key Results
- Massive Reduction: Speculative naming (HARD mode) reduced the Slop Ratio by 95% on average across all models.
- Collision Probability: The mean probability of a name collision dropped from 7.3% to 0.4%.
- Zero Known Slop: The method successfully held collisions with "known" slop names at zero.
- Human Preference: In blinded evaluations, human readers preferred the names generated by speculative naming over baseline outputs, finding them more natural and creative.
- Cost-Effectiveness: While it requires an extra model call, the method is budgetable and significantly more effective per token than multi-agent reviewer strategies.
Limitations and Future Work
The authors acknowledge several areas for further exploration:
- Beyond Names: While the paper focuses on names, "slop" also exists in sensory clichés ("a shimmer of light") and formulaic openings ("In a world where..."). The authors provide a proof-of-concept for sentence openers but leave full mitigation to future work.
- Cold Start: The runtime tracker needs a "warm-up" period (about 50 names) before it can effectively identify and penalize new emergent slop.
- Discourse Level: The current method doesn't handle how characters are referred to throughout a story (e.g., alternating between full name, first name, and pronouns), which remains a challenge for LLM consistency.
Summary updated