Estranged Predictions: Measuring Semantic Category Disruption with Masked Language Modelling

Yuxuan Liu, Haim Dubossarsky, Ruth AhnertarXiv 2025 · DOI · arXiv

Introduction

Science fiction (SF) has long been celebrated for its capacity to blur the boundaries between humans, animals, and machines. Literary theorists like Darko Suvin argue that the genre operates through "cognitive estrangement"—the process of making the familiar strange to challenge societal assumptions. However, large-scale quantitative evidence for this linguistic destabilization has been scarce. This study addresses this gap by using Masked Language Models (MLMs) to measure "conceptual permeability," investigating whether SF reorganizes the linguistic scaffolding of identity differently than general fiction.

Methodology

The researchers developed a robust three-stage pipeline to quantify how porous semantic categories are within different literary contexts.

The three-stage pipeline for measuring conceptual permeability, from sentence extraction to semantic classification.
  1. Sentence Extraction & Masking: Sentences containing the terms human, animal, and machine were extracted from two major corpora: the Gollancz SF Masterworks (SF) and the NovelTM dataset (General Fiction). The target terms were replaced with a [MASK] token.
  2. Masked Language Modelling (MLM): RoBERTa-base was employed to predict the top 5 most likely substitutes for the masked tokens, serving as a proxy for the linguistic expectations embedded in the text's context.
  3. Semantic Classification: To interpret the raw token outputs, the Gemini API acted as a zero-shot classifier, categorizing predictions into a specific taxonomy (e.g., classifying "robot" as a Machine or Hybrid).

The study evaluated these predictions using metrics such as Retention Rate (frequency of same-category predictions), Replacement Rate (cross-category shifts), and Entropy (contextual uncertainty).

Key Results and Analysis

The analysis revealed a striking contrast between the volatility of technological categories and the resilience of the human category.

Retention and cross-category replacement patterns comparing the Science Fiction Corpus (top) and the General Fiction/NovelTM Corpus (bottom).

The Fragility of the Machine

The "machine" category proved to be the most unstable in SF. RoBERTa was significantly less likely to suggest machine-related substitutes in SF contexts (25.8% retention) compared to general fiction (30.2%). This suggests that SF narratives actively dismantle the traditional "machinic" identity, often replacing it with human referents—a phenomenon known as anthropomorphism. Machine-to-human substitutions were 38.8% higher in SF than in general fiction.

Anthropocentric Inertia

Despite SF's thematic radicalism, the "human" category remains a stable anchor. Retention for human terms showed no statistically significant difference between genres (54.0% in SF vs. 56.5% in general fiction). This suggests that English remains deeply "human-coded," making it difficult for even the most imaginative SF to fully unseat the human as the primary linguistic reference point.

Zoomorphism and Entropy

SF also demonstrated a higher tendency toward "zoomorphism," where human terms are replaced by animal ones, signaling a subtle decentering of the human subject. Furthermore, the entropy for machine referents was significantly higher in SF (1.852) than in general fiction (1.743), indicating that SF provides a broader and more uncertain "substitutional latitude" for technology.

Conclusion

This research provides empirical evidence that science fiction significantly destabilizes the linguistic boundaries of machines and animals. While the "human" remains a surprisingly resilient linguistic category, SF's ability to increase the permeability of other identities confirms its role as a genre of cognitive estrangement. Future work may involve fine-tuning models on specific genre corpora to uncover deeper levels of semantic disruption and expanding the taxonomy to include specific tropes like AI and cyborgs.

Summary updated

BibTeX
@misc{liu2025estranged,
  author = {Yuxuan Liu and Haim Dubossarsky and Ruth Ahnert},
  title = {{Estranged Predictions: Measuring Semantic Category Disruption with Masked Language Modelling}},
  year = {2025},
  eprint = {2511.08109},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.08109},
}
Figure