MirrorStories: Reflecting Diversity through Personalized Narrative Generation with Large Language Models

Sarfaroz Yunusov, Hamza Sidat, Ali EmamiEMNLP 2024 · DOI

Abstract

This paper introduces MIRRORSTORIES, a novel corpus of 1,500 personalized short stories generated by Large Language Models (LLMs) to address the persistent "representation gap" in literature. By weaving specific identity traits—including ethnicity, gender, and niche interests—into narratives based on classic morals, the study demonstrates that LLMs can generate content that significantly outperforms generic stories in engagement and satisfaction. Our findings suggest that personalization not only improves reader connection but also enhances the linguistic diversity of the generated text.

Introduction

Literature has long struggled with a lack of diversity; data from the Cooperative Children’s Book Center (CCBC) reveals a significant underrepresentation of non-white minority groups. "Mirror books"—stories where readers see their own identities and cultures reflected—are essential for fostering belonging and improving educational outcomes. However, creating high-quality, personalized content at scale is traditionally labor-intensive. This research investigates whether LLMs can bridge this gap by generating "mirror stories" that maintain narrative quality and moral coherence while reflecting the reader's unique identity.

Methods and Evaluation Framework

The researchers developed a framework using top-tier LLMs, including GPT-4, Claude-3 Sonnet, and Gemini 1.5 Flash, to generate stories of 250–300 words. Each story was grounded in a moral from Aesop’s Fables and tailored to specific identity elements (name, age, gender, ethnicity, and interest).

To ensure the effectiveness of this personalization, the study employed a dual evaluation strategy. A "Personalization Validation" experiment tested whether humans and LLMs could identify hidden traits within the text. Subsequently, a "Personalization Impact" experiment involved 26 diverse human judges who rated the stories across four key metrics: Satisfaction, Quality, Engagement, and Personalness.

The evaluation framework utilized both LLM and human evaluators to assess story quality and the effectiveness of identity integration.

Key Insights

  • Seamless Integration: LLMs successfully integrated identity traits through context (e.g., cultural nuances or traditional attire) rather than explicit mentions. Human evaluators identified gender and ethnicity with 100% and 94% accuracy, respectively, even in "seamless" modes.
  • Linguistic Richness: Using the Shannon Diversity Index (SDI), the study found that personalization actually increased textual diversity. The requirement to include specific details (like "Uzbek carpentry") pushed models into less common regions of their latent space, resulting in more unique prose.
  • Evaluator Bias: The study noted that GPT-4 exhibited preferential biases, such as giving higher "personalness" ratings to non-binary characters and favoring certain ethnicities like Norwegian or Japanese.

Results

The MIRRORSTORIES corpus demonstrated superior performance over generic baselines. Personalized LLM stories received an average engagement rating of 4.22/5, compared to just 3.37/5 for generic human-written or LLM-generated stories.

Personalized stories consistently achieved high scores across all qualitative metrics, significantly outperforming generic baselines in engagement and personal resonance.

Discussion and Limitations

While the results are promising, the study identified several areas for future work. The current scope was limited to 300-word stories; it remains to be seen if LLMs can maintain this level of personalization in long-form novels. Additionally, the human evaluators were primarily university students, suggesting a need for broader demographic testing. Finally, the researchers propose exploring "window stories"—investigating how stories personalized for one individual might build empathy in "out-group" readers.

Conclusion

MirrorStories proves that LLMs are capable of generating deeply personalized, high-quality narratives that resonate with diverse readers. This framework offers a scalable solution to the representation gap, potentially transforming how inclusive literature is produced and consumed.

Summary updated

BibTeX
@inproceedings{yunusov2024mirrorstories,
  author = {Sarfaroz Yunusov and Hamza Sidat and Ali Emami},
  title = {{MirrorStories: Reflecting Diversity through Personalized Narrative Generation with Large Language Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.382},
}
Figure