Learning and Evaluating Character Representations in Novels

Naoya Inoue, Charuta Pethe, Allen Kim, Steven SkienaACL 2022 · OpenReview

Abstract

We address the problem of learning fixed-length vector representations of characters in novels. Recent advances in word embeddings have proven successful in learning entity representations from short texts, but fall short on longer documents because they do not capture full book-level information. To overcome the weakness of such text-based embeddings, we propose two novel methods for representing characters: (i) graph neural network-based embeddings from a full corpus-based character network; and (ii) low-dimensional embeddings constructed from the occurrence pattern of characters in each novel. We test the quality of these character embeddings using a new benchmark suite to evaluate character representations, encompassing 12 different tasks. We show that our representation techniques combined with text-based embeddings lead to the best character representations, outperforming text-based embeddings in four tasks. Our dataset and evaluation script will be made publicly available to stimulate additional work in this area.

Introduction

In natural language processing, representing entities through vector embeddings has become standard practice. Most models, such as Word2Vec or BERT, rely on the Distributional Hypothesis, which posits that words appearing in similar contexts share similar meanings. However, these models typically operate within a narrow "myopic" window of surrounding text. For literary analysis, this local focus is insufficient. To truly understand a character, a model must capture global information: the genre of the novel, the author's style, and the character's interactions across an entire corpus.

Methodology

To address the limitations of local context, we propose two novel methods to supplement standard text-based embeddings:

  1. Graph-based Embeddings (gr): We construct a comprehensive character network where nodes represent characters, books, authors, and contextual action words (e.g., "traded", "ran"). By applying DeepWalk—a technique using random walks to learn node relationships—the model learns that characters are similar if they share authors, genres, or interaction patterns.
Graph-based representation schema linking authors, books, characters, and actions.
  1. Positional Embeddings (pos): Narrative arcs often define character roles. We divide each novel into ten segments and map the frequency of a character's appearances across these segments to create a "narrative signature."
  2. Hybrid Approach: These features are integrated with Context-aggregated Embeddings (using RoBERTa) to capture both local descriptive nuances and global structural information.

Evaluation Framework

We introduce the Character Embedding Benchmark (CEB), a suite of 12 tasks designed to evaluate representations at three levels:

  • Character-level: Gender, Role (e.g., "aristocrat"), Protagonist status, and Identity (linking the same character across different books).
  • Context-level: Cloze tests, Speaker identification, and Question Answering.
  • Book-level: Predicting author, title, or genre based on character vectors.

The training data comprises a massive collection of 17,275 books from Project Gutenberg.

Results and Discussion

Our hybrid model (rb_ag + gr + pos) significantly outperformed standalone text-based models in most categories. Key insights include:

  • Archetypal Clustering: The embedding space naturally clusters characters with similar roles, such as "kings," "queens," and "barons," even without explicit labels for these archetypes.
t-SNE visualization showing the emergence of archetypal clusters in the embedding space.
  • Identity Linking: The model achieved 97.8% accuracy in identifying the same character across different books, demonstrating its ability to learn "universal" representations regardless of specific plot variations.
  • Metadata Prediction: The graph-based approach reached 85.6% accuracy in author identification and 95.5% for book titles, proving that character interactions serve as a unique "fingerprint" for specific works.

Conclusion

While our approach shows significant gains, challenges remain. Preprocessing noise from Named Entity Recognition in archaic texts and the static nature of fixed vectors—which fail to capture dynamic character development—are areas for future research. Nonetheless, this work provides a robust framework for deeper literary analysis through computational linguistics.

Summary updated

Abstract

We address the problem of learning fixed-length vector representations of characters in novels. Recent advances in word embeddings have proven successful in learning entity representations from short texts, but fall short on longer documents because they do not capture full book-level information. To overcome the weakness of such text-based embeddings, we propose two novel methods for representing characters: (i) graph neural network-based embeddings from a full corpus-based character network; and (ii) low-dimensional embeddings constructed from the occurrence pattern of characters in each novel. We test the quality of these character embeddings using a new benchmark suite to evaluate character representations, encompassing 12 different tasks. We show that our representation techniques combined with text-based embeddings lead to the best character representations, outperforming text-based embeddings in four tasks. Our dataset and evaluation script will be made publicly available to stimulate additional work in this area.
BibTeX
@inproceedings{inoue2022learning,
  author = {Naoya Inoue and Charuta Pethe and Allen Kim and Steven Skiena},
  title = {{Learning and Evaluating Character Representations in Novels}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
  url = {https://openreview.net/forum?id=ETl4NBwtar},
}
Figure