Unsupervised Authorship Attribution for Medieval Latin Using Transformer-Based Embeddings

Loic De Langhe, Orphee De Clercq, Veronique HosteCOLING 2024

TL;DR

This paper explores using transformer-based language models to automatically attribute authorship of medieval Latin texts without needing pre-labeled data. They demonstrate that these modern NLP techniques can effectively cluster texts by author, offering a promising unsupervised approach for analyzing historical documents where traditional methods are challenging.

What Problem Does This Paper Address?

Attributing authorship to historical texts, especially those from periods with less linguistic standardization like medieval Latin, is a long-standing challenge. Traditionally, this involves labor-intensive stylometric analysis, often relying on handcrafted linguistic features. While modern NLP has made strides in authorship attribution for well-resourced languages, its application to ancient and medieval texts, which often lack large, labeled datasets, remains underexplored. This paper tackles this gap by investigating whether advanced transformer models, capable of learning rich contextual representations, can be leveraged for unsupervised authorship attribution in medieval Latin, thereby reducing reliance on manual feature engineering and large labeled corpora.

Methods

The core of this research lies in using transformer-based language models to generate "embeddings" – numerical representations of text that capture semantic and stylistic nuances. Specifically, the authors employed four different transformer models:

  • Monolingual Latin RoBERTa: A model specifically trained on Latin text.
  • Multilingual mBERT: A general-purpose multilingual model.
  • Multilingual DeBERTaV3: A more advanced multilingual model known for strong performance.
  • Longformer: A model designed to handle much longer texts than standard BERT-like models, crucial for medieval documents which can be extensive.

These embeddings were then fed into two unsupervised clustering algorithms:

  • Agglomerative Clustering: A hierarchical method that progressively merges clusters. This was used for quantitative evaluation of clustering quality.
  • Self-Organizing Map (SOM): A neural network that creates a low-dimensional, topological representation of the data, allowing for more interpretable visualization of the clusters.

The embeddings were created by concatenating the last four encoder layers of the transformers, resulting in a 3072-dimensional vector for each document.

Datasets

The study utilized two existing corpora of medieval Latin texts:

  • MedLatin1: Composed of medieval Latin epistles.
  • MedLatin2: Composed of medieval Latin literary analyses.

These corpora were merged into a single dataset. After removing texts with unknown authors, the final dataset comprised 293 documents from 22 distinct authors. This is a newly combined benchmark for this specific task.

Key Insights and Ideas

  • Transformer Power for Historical Texts: The paper strongly suggests that modern transformer models, particularly those designed for long documents, can effectively capture stylistic signatures in medieval Latin, even without explicit training on this specific task.
  • Long Document Models Shine: The Longformer model, capable of processing longer texts, yielded the best clustering results, highlighting the importance of context length for capturing authorial style in extensive historical documents.
  • Unsupervised Learning is Viable: The success of unsupervised clustering methods demonstrates that we can identify authorial groupings without needing pre-existing, manually curated labels, which are scarce for historical texts.
  • SOMs Offer Interpretability: The Self-Organizing Maps provided a valuable visual tool, allowing researchers to see how authors' works cluster together and identify potential outliers or distinct stylistic groups.

Key Results

  • Quantitative Evaluation: The Longformer model achieved the highest scores across intrinsic evaluation metrics (Silhouette Coefficient and Calinski-Harabasz Index) and a strong score for the extrinsic metric (Rand Index), indicating superior clustering quality.
  • Extrinsic vs. Intrinsic Performance: Interestingly, the monolingual Latin RoBERTa model performed well on the extrinsic evaluation (matching ground truth) but poorly on intrinsic measures, suggesting its clusters were distinct but very similar to each other.
  • Qualitative Visualization: The SOM visualizations showed clear, albeit sometimes overlapping, groupings of authors. Notably, the works of Guido Faba appeared highly distinct, and the SOM could even highlight specific instances where a text by one author was closer to another's work, suggesting potential misattribution or influence.

Limitations and Future Work

  • Limitations: The authors acknowledge that while unsupervised methods are promising, close reading and traditional feature-based methods are still essential for definitive authorship attribution. The intrinsic evaluation metrics for unsupervised methods can be challenging to interpret definitively.
  • Future Work: The paper suggests that these transformer-based embeddings can serve as a valuable "early diagnostic tool" for outlier detection and narrowing down the search space within large medieval document collections. Further research could explore more advanced unsupervised techniques or hybrid approaches combining transformer embeddings with traditional stylometric features.

Summary updated

BibTeX
@inproceedings{langhe2024unsupervised,
  author = {Loic De Langhe and Orphee De Clercq and Veronique Hoste},
  title = {{Unsupervised Authorship Attribution for Medieval Latin Using Transformer-Based Embeddings}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2024},
}
Figure