Sameness entices, but novelty enchants in fanfiction online

Elise Jing, Simon DeDeo, Devin Robert Wright, Yong‐Yeol Ahn

TL;DR

This paper challenges the "balance theory" of cultural success by analyzing fanfiction. It finds that while novelty enhances enjoyment, familiarity drives initial attraction, leading to a U-shaped curve for overall success, not the expected inverse U-shape. This suggests cultural evolution is a tug-of-war between the desire for the familiar and the allure of the new.

What Problem Does This Paper Address?

The prevailing theory in cultural evolution is the "balance theory," which posits that successful cultural products strike a balance between familiarity and novelty. This theory suggests that people prefer content that is recognizable but not predictable, and novel but not so alien as to be off-putting. While this idea has intuitive appeal and some empirical support across various domains like music, film, and advertising, it's difficult to isolate the effect of novelty from confounding factors such as marketing, price, and genre expectations. The authors aim to test this theory in a controlled environment to disentangle the true relationship between novelty and success.

Methods

The researchers employed a data-driven approach using fanfiction, a rich dataset that naturally controls for many confounding variables. They used Latent Dirichlet Allocation (LDA) to model the novelty of fanfiction works. Specifically, they calculated a "topic novelty score" for each piece by measuring its distance in a topic-based feature space from the centroid of previously published works within the same fandom. They then used multiple regression models and Generalized Additive Models (GAMs) to analyze the relationship between this novelty score and various metrics of success, while controlling for factors like fandom, author history, and work characteristics.

Datasets

The primary dataset used in this study is a large collection of fanfiction from the online archive Archive of Our Own (AO3). The authors collected approximately 671,908 fanfiction pieces across 23 different fandoms, published between January 2010 and March 2016. This dataset is notable for its scale and the availability of detailed metadata, allowing for robust analysis.

Key Insights and Ideas

  • Familiarity drives attraction, novelty drives enjoyment: The study reveals a crucial dichotomy: while novelty might make a piece more enjoyable once read, it's familiarity that initially draws readers in. This explains why overall success metrics (like views) might decrease with novelty.
  • The "balance theory" is too simplistic: The expected inverse U-shaped curve (peak success at moderate novelty) was not observed for overall success. Instead, a U-shaped curve emerged, suggesting that both very familiar and somewhat novel works can achieve success, but highly novel works tend to be overlooked initially.
  • Fanfiction as a controlled experiment: The unique nature of fanfiction—being freely available, community-driven, and based on existing canons—provides a cleaner environment to study cultural evolution than traditional media, mitigating issues like marketing bias and price.
  • Cultural evolution as a battle against inertia: The findings suggest that cultural evolution might be a process of overcoming inertia, where the tendency to consume the familiar is a powerful force that novelty must contend with.

Key Results

  • Overall success declines with novelty: Across metrics like "Kudos" (likes), "hits" (views), "comments," and "bookmarks," the study found a generally negative correlation with novelty. As novelty increased, these engagement metrics tended to decrease.
  • U-shaped curve for overall success: When considering both low and high novelty, the models revealed a U-shaped relationship for overall success metrics, indicating that both highly familiar and moderately novel works perform better than highly novel ones in terms of initial attraction.
  • Inverted U-shaped curve for enjoyment: The "Kudos-to-hits ratio" (a proxy for enjoyment per reader) showed an inverted U-shaped curve. This means that while fewer people might read highly novel works, those who do tend to enjoy them more, supporting the idea that novelty enhances enjoyment.
  • Term-level vs. Topic-level novelty: While both methods of measuring novelty showed similar trends, topic-level novelty (based on LDA topic distributions) revealed a stronger effect for the "optimal differentiation phenomena."

Limitations and Future Work

  • Correlational data: The study acknowledges that its observational data design prevents drawing definitive causal conclusions.
  • Subculture bias: The findings are based on fanfiction, a subculture often dominated by young, female, English-speaking audiences. Generalizing these results to other cultural products or demographics requires caution.
  • Novelty measurement: The methods for quantifying novelty (topic and term-based) capture content but may miss nuances in narrative arrangement, style, or semantic meaning.
  • Future work: The authors suggest exploring causal inference study designs and investigating other quantitative methods for estimating novelty in cultural products. They also propose that their analytical strategies could be applied to other forms of text narrative data.

Summary updated

Abstract

Cultural evolution is driven by how we choose what to consume and share with others. A common belief is that the cultural artifacts that succeed are the ones that balance novelty and conventionality. This “balance theory” suggests that people prefer works that are familiar, but not so familiar as to be boring; novel, but not so novel as to violate the expectations of their genre. We test this idea using a large dataset of fanfiction, a unique data source that mitigates many common critical shortcomings in the study of creative works. We apply a multiple regression model and a generalized additive model to examine how the recognition a work receives varies with its novelty, estimated through a Latent Dirichlet Allocation topic model, in the context of existing works. We find the opposite pattern of what the balance theory predicts—overall success declines almost monotonically with novelty and exhibits a U-shaped instead of an inverse U-shaped curve. This puzzle is resolved by teasing out two competing forces: sameness attracts the masses whereas novelty provides enjoyment. Taken together, even though the balance theory holds in terms of expressed enjoyment, the overall success can show the opposite pattern due to the dominant role of familiarity to attract the audience. Under these two “forces”, cultural evolution may have to work against inertia—the appetite for consuming the familiar—and may resemble a punctuated equilibrium, marked by occasional leaps.
BibTeX
@inproceedings{jing2025sameness,
  author = {Elise Jing and Simon DeDeo and Devin Robert Wright and Yong‐Yeol Ahn},
  title = {{Sameness entices, but novelty enchants in fanfiction online}},
  booktitle = {Humanities},
  year = {2025},
  doi = {10.1057/s41599-025-05166-3},
}
Figure