Towards Automatic Variant Analysis of Ancient Devotional Texts

Amir Hazem, Béatrice Daille, Dominique Stutzmann, Jacob Currie, Christine JacquinACL 2019

TL;DR

This paper introduces a novel linguistic classification for analyzing textual variants in ancient devotional texts, specifically the "Obsecro Te" prayer from Books of Hours. By applying this classification to a large dataset, the authors evaluate several unsupervised machine learning approaches for automatically extracting and categorizing these variants, paving the way for large-scale historical linguistic analysis.

What Problem Does This Paper Address?

The study of historical texts, particularly those from the medieval period, often involves understanding how texts evolved and varied over time and across different regions. Books of Hours, popular devotional texts from the late Middle Ages, are a prime example. While these texts were widely used, their manual transcription and analysis for variants have been limited due to the sheer volume of material and the lack of readily available digital transcriptions. Previous attempts to categorize variants in texts like the "Obsecro Te" prayer relied on simple surface-level observations (addition, substitution, omission) without incorporating linguistic insights. This paper addresses the challenge of automatically and systematically analyzing the vast number of textual variants within these historical documents, aiming to uncover patterns of linguistic change and transmission.

Methods

The core of the paper lies in its proposed linguistic classification of variants. Instead of relying solely on edit operations, the authors introduce a typology that considers both "synforms" (words with similar forms) and conceptual similarities. This classification includes:

  • Orthography: Variations in spelling (e.g., "dilecto" vs. "delecto").
  • Inflexion: Changes in grammatical endings (e.g., "crucem" vs. "cruce").
  • Derivation: Creation of new words through affixation (e.g., "dilecto" vs. "dilectissimo").
  • Lexical Substitution: Replacing one word with another, including synonyms or near-synonyms (e.g., "tribuas" vs. "concedas").
  • Expansion: Adding elements to a phrase or sentence, often for emphasis or clarification (e.g., "criminalibus peccatis" vs. "criminalibus peccatis vel mortalibus").
  • Reduction: Removing elements from a phrase or sentence (e.g., "ostendem michi gloriosam" vs. "ostendem michi").
  • Permutation: Rearranging the order of words within a phrase (e.g., "criminalibus peccatis" vs. "peccatis criminalibus").

To automatically extract these variants, the paper evaluates four unsupervised state-of-the-art approaches:

  • Edit Distance (Levenshtein): Measures the minimum number of single-character edits required to change one word/ngram into another.
  • Jaccard Index: Measures the similarity between two sets (in this case, character sets of ngrams) by dividing the size of their intersection by the size of their union.
  • Distributional Bag of Words (BoW): Represents words/ngrams by their context vectors and measures similarity using cosine similarity.
  • Word Embeddings (Word2Vec, FastText): Represents words/ngrams as dense vectors in a continuous vector space, capturing semantic relationships.

Datasets

The primary dataset used is the Beyond Use database, which contains manually extracted variants from 772 manuscripts of the "Obsecro Te" prayer. This dataset comprises over 21,000 variants, with 3,298 distinct variants, segmented into 49 passages. For evaluation, the authors created four distinct lists of variants based on ngram size (unigrams, bigrams, trigrams, fourgrams) and a combined list of all ngram sizes.

Key Insights and Ideas

  • Linguistic Classification is Crucial: The paper strongly advocates for a linguistic approach to variant analysis over purely edit-distance-based methods, as it provides a richer understanding of textual evolution.
  • No Single Best Method: Different unsupervised methods excel at detecting different types of variants. Edit distance is good for simple orthographic and inflectional changes, while Jaccard Index and word embeddings are better suited for permutations and lexical substitutions, respectively.
  • Context Matters for Variants: The "Obsecro Te" prayer, despite its popularity, exhibits significant variation across manuscripts, confirming that even common texts were not copied identically.
  • Rare Variants as Potential Errors or Innovations: The analysis distinguishes between rare variants (appearing in very few copies) and frequent variants. Rare variants might indicate misspellings or unique scribal choices, while frequent variants suggest established variations.
  • Temporal and Geographical Patterns Exist: While not the primary focus of the evaluation, the paper hints at the potential to uncover temporal and geographical correlations in variant usage, which could shed light on the transmission and localization of texts.

Key Results

  • Edit Distance performed best for unigram variants but struggled with longer ngrams, particularly permutations.
  • Jaccard Index showed better performance for ngrams greater than 1, effectively handling permutations.
  • The Distributional Bag of Words (BoW) approach using log-likelihood achieved the best results on the entire evaluation list (ALL), indicating its robustness for variable-length variants.
  • Word Embeddings (W2V and FastText) showed strong performance for ngrams greater than 1, suggesting their utility for capturing lexical substitutions.
  • A linear combination of approaches did not yield significant improvements.
  • The study found that rare variants (appearing in less than five copies) are common, suggesting a high degree of scribal freedom or error.
  • Inflectional variants were observed as both rare and frequent, highlighting the dynamic nature of grammatical forms.

Limitations and Future Work

  • Exhaustiveness of Typology: The authors acknowledge that their linguistic typology is not exhaustive and can be extended to include other operations like composition.
  • Handling Complex Variants: Some phenomena, such as expansion and reduction variants of variable length, remain difficult to handle with current methods.
  • Distinguishing Variant Types: Further work is needed to reliably distinguish between orthographic, inflectional, and derivational variants automatically.
  • Word Embedding Limitations: Pre-trained word embedding models may not contain all words from historical corpora, and their performance on unigrams can be limited.
  • Future Directions: The authors suggest that their methodology can be a basis for analyzing the content of Books of Hours more broadly, uncovering temporal and geographical correlations, and guiding experts in analyzing such phenomena. They also propose extending the analysis to other linguistic operations and combining approaches for better results.

Summary updated

BibTeX
@inproceedings{hazem2019towards,
  author = {Amir Hazem and Béatrice Daille and Dominique Stutzmann and Jacob Currie and Christine Jacquin},
  title = {{Towards Automatic Variant Analysis of Ancient Devotional Texts}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure