Graph convolutional networks for exploring authorship hypotheses

Tom LippincottNAACL 2019

TL;DR

This paper applies graph convolutional networks (GCNs) to the centuries-old task of source criticism in the Hebrew Bible, specifically the Documentary Hypothesis. They demonstrate that GCNs, by leveraging the document's inherent structure, significantly outperform traditional models in identifying textual sources, highlighting the importance of structural information in literary analysis.

What Problem Does This Paper Address?

For over 150 years, scholars have debated the composition of the first five books of the Hebrew Bible (the Torah), proposing that they are a composite work of multiple distinct sources (the Documentary Hypothesis or DH). This hypothesis suggests that different authors or editorial schools contributed to the text, each with their own stylistic tendencies. The challenge for computational linguistics has been to computationally identify these sources based on linguistic features, a task akin to authorship attribution. Traditional methods often treat documents as isolated units, ignoring their internal structure. However, the DH implies that sources might manifest themselves through structural elements, like inserting entire narrative sections or making edits at specific granularities. This paper aims to bridge this gap by developing models that can exploit the hierarchical, tree-like structure of the Torah to better identify its constituent sources.

Methods

The authors employ Graph Convolutional Networks (GCNs), a deep learning architecture designed to operate on graph-structured data. Unlike standard neural networks (like MLPs) that process feature vectors independently, GCNs allow nodes (in this case, text spans) to aggregate information from their neighbors. This is achieved by using an adjacency matrix that encodes relationships between text spans.

Key methodological points:

  • GCN Architecture: GCNs iteratively update node representations by combining their own features with those of their neighbors, controlled by the adjacency matrix. The depth of the GCN (number of layers) determines how far information can propagate.
  • Feature Engineering: To avoid trivial solutions based on topic or open-class vocabulary, they focus on function-morphs (stems, prefixes, suffixes) that are common across sources. They also introduce two structural features:
  • Granularity Indicators: Binary features indicating the level of a text span (e.g., chapter, verse, token).
  • Sibling Adjacency Matrix: A matrix representing relationships between spans that share the same parent in the document's tree structure.
  • Baseline Models: They compare GCNs against logistic regression (LR) and multi-layer perceptrons (MLPs), which do not inherently leverage document structure.

Datasets

The primary dataset used is the Westminster Leningrad Codex (WLC), a publicly available TEI-encoded representation of the oldest complete Masoretic text of the Hebrew Bible.

  • WLC: This dataset is annotated with source labels according to the Documentary Hypothesis, as described by Friedman (2003).
  • Data Structure: The WLC is a tree-structured document comprising books, chapters, verses, tokens, and morphs. The paper uses labeled spans (fragments of the Torah) as data points.
  • Preprocessing: Vowel pointing, introduced later, was removed. The text was segmented into morphs, and a reduced set of 70 common morphs was used for feature extraction.
  • Splitting: The labeled spans were randomly split into 80% training, 10% development, and 10% testing sets.

Key Insights and Ideas

  • Structure Matters: The most significant insight is that incorporating structural information about the text (granularity and sibling relationships) dramatically improves performance, especially for models with hidden layers. This suggests that the DH is not just about isolated stylistic choices but also about how different sources are integrated into the larger narrative structure.
  • GCNs Excel: GCNs, by design, are well-suited to exploit this structural information, leading to superior performance compared to structure-unaware models like MLPs and LR.
  • Granularity is Crucial: The granularity indicator features were particularly impactful, boosting performance by 10-20 points for GCNs and MLPs. This implies that the size of textual units attributed to a source is a strong signal.
  • Beware of Topic Bias: When using features that capture topic (like a full vocabulary bag-of-words), adding granularity features can actually hurt performance, suggesting that models might overfit to topic-specific cues rather than stylistic ones.

Key Results

The paper's primary result is that the GCN with ReLU activation and granularity features achieved the highest macro F-score of 48.60% on the test set. This significantly outperforms:

  • Logistic Regression: 45.80% (without granularity)
  • MLP with ReLU and granularity: 47.45%
  • GCN with linear activation and granularity: 46.64%

The authors note that while these scores might seem modest, they represent a significant improvement over structure-unaware models for this complex task.

Limitations and Future Work

  • Limited Dataset Size: The authors acknowledge that the dataset is "very small," which influences their training strategy (treating it as a single large batch) and may limit the generalizability of findings.
  • Granularity Feature Overfitting: They observed that granularity features can lead to "particularly damaging overfitting" when combined with topic-capturing features, and plan further investigation into this.
  • Future Directions:
    • Other Graph Models: Explore more advanced graph neural network architectures like graph attention networks (GATs) and tree-structured VAEs.
  • Augmented GCNs: Investigate augmented GCNs for better integration of information.
  • Broader Hypotheses: Extend the approach to other scholarly hypotheses and historical texts.
  • Unsupervised Resources: Experiment with training unsupervised distributional models on Biblical texts to create less biased linguistic resources, avoiding reliance on potentially biased scholarly interpretations.
  • Feature Engineering: Incorporate richer features like sentiment analysis, concept realization (e.g., Elohim vs. Jehovah), and relationships between words.

Summary updated

BibTeX
@inproceedings{lippincott2019graph,
  author = {Tom Lippincott},
  title = {{Graph convolutional networks for exploring authorship hypotheses}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2019},
}
Figure