Restoring ancient text using deep learning: a case study on Greek epigraphy

Yannis M. Assael, Thea Sommerschield, Jonathan PragEMNLP 2019 · DOI

TL;DR

This paper introduces PYTHIA, a deep learning model that automatically restores damaged ancient Greek inscriptions, significantly outperforming human epigraphists in accuracy. By leveraging a novel dataset and a sophisticated sequence-to-sequence architecture, PYTHIA offers a powerful assistive tool for digital humanities, paving the way for more efficient and accurate analysis of historical texts.

What Problem Does This Paper Address?

Ancient History heavily relies on epigraphy, the study of ancient inscribed texts, for direct evidence of past civilizations. However, many inscriptions are damaged, making parts of the text illegible. Epigraphists traditionally restore these missing sections by manually searching vast repositories for similar texts and contextual clues – a time-consuming and often subjective process. This manual approach can be hindered by minor variations in search queries, making it difficult to assess the probability of different restorations. The paper addresses the need for an automated, data-driven method to assist epigraphists in this challenging restoration task.

Methods

The core of the paper is the PYTHIA model, a sequence-to-sequence deep learning architecture. It employs a Long-Short Term Memory (LSTM) encoder-decoder framework with an attention mechanism. This allows the model to focus on relevant parts of the input text when generating the output, crucial for handling long-term dependencies in inscriptions.

A key innovation is PYTHIA's ability to process text at both character and word levels simultaneously. This is achieved by concatenating character embeddings with word embeddings for the 100,000 most frequent words in the dataset. This dual-level processing helps overcome the challenge of dealing with partially legible words, where character-level information is crucial but word-level context is also vital. The authors also explored a bidirectional LSTM variant (PYTHIA-BI-WORD) for improved context understanding.

Datasets

The primary dataset used is PHI-ML, a newly created machine-actionable version of the Packard Humanities Institute's (PHI) digital corpus of ancient Greek inscriptions. PHI-ML was derived from the existing PHI corpus by developing a pipeline to convert edited texts into a format suitable for machine learning. This involved standardizing the alphabet, introducing special characters for missing ('-') and predicted ('?') characters, and cleaning the data. The dataset comprises over 3.2 million words. The inscriptions were split into training, validation, and test sets.

Key Insights and Ideas

  • Deep Learning for Epigraphy: The paper demonstrates the significant potential of deep learning to automate and enhance a traditionally manual and expert-driven field like epigraphy.
  • Dual-Level Text Processing: Combining character-level and word-level embeddings is a crucial insight for handling the fragmented nature of ancient texts, capturing both fine-grained details and broader context.
  • Context is King: The model's performance, particularly its Top-20 accuracy, significantly improves with longer input context lengths, highlighting the importance of surrounding text for accurate restoration.
  • Assistive Technology: PYTHIA is positioned not as a replacement for human epigraphists, but as a powerful assistive tool, providing multiple hypotheses and confidence levels to guide human decision-making.

Key Results

  • Character Error Rate (CER): PYTHIA-BI-WORD achieved a CER of 30.1%, a substantial improvement over human epigraphists' 57.3% CER.
  • Top-20 Accuracy: In 73.5% of cases, the ground-truth sequence was among PYTHIA-BI-WORD's Top-20 predictions, demonstrating its utility in providing relevant suggestions.
  • Ablation Studies: The bidirectional LSTM and word embeddings (PYTHIA-BI-WORD) significantly outperformed simpler models like PYTHIA-UNI (unidirectional LSTM with only characters), showing the effectiveness of the proposed architecture.
  • Context Length Impact: Predictive performance (measured by Top-20 accuracy) showed a positive correlation with context length, peaking around 500 characters.

Limitations and Future Work

  • Language Specificity: The current model is trained on ancient Greek epigraphy. While the architecture is designed to be generalizable, further work would be needed to adapt it to other languages and scripts.
  • Data Availability: The performance of deep learning models is heavily dependent on the availability of large, high-quality datasets. The creation of PHI-ML is a significant step, but similar efforts for other epigraphic corpora would be beneficial.
  • Human-in-the-Loop: While PYTHIA provides excellent suggestions, the paper acknowledges that human expertise remains crucial for final interpretation and validation. Future work could explore more sophisticated human-AI collaboration interfaces.
  • Generalization to Other Text Types: The authors suggest that PYTHIA could be applied to other disciplines dealing with ancient texts, such as philology, papyrology, and codicology, and to any language, ancient or modern.

Summary updated

BibTeX
@inproceedings{assael2019restoring,
  author = {Yannis M. Assael and Thea Sommerschield and Jonathan Prag},
  title = {{Restoring ancient text using deep learning: a case study on Greek epigraphy}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/d19-1668},
}
Figure