A new machine-actionable corpus for ancient text restoration

Will Fitzgerald, Justin BarneyACL 2024

TL;DR

Researchers have developed the Machine-Actionable Ancient Text (MAAT) Corpus, a novel dataset designed to train and evaluate machine learning models for restoring damaged ancient texts. This corpus provides a machine-readable format of ancient Greek, Latin, and Coptic texts, including lacunae and their restorations, enabling more realistic evaluation of AI systems compared to previous methods.

What Problem Does This Paper Address?

Ancient texts, particularly those on papyri and inscriptions, are often damaged, leading to gaps or "lacunae" in the text. The painstaking task of accurately transcribing and restoring these texts is crucial for historical and linguistic research. While computational approaches, especially deep learning and large language models (LLMs), show promise in assisting this task, they require substantial amounts of training data and robust evaluation methods. Existing corpora, while valuable, often contain rich metadata and complex structures that are not directly machine-actionable for LLMs. Furthermore, evaluation methods that rely on artificially introduced gaps don't fully capture the complexities of real-world text restoration, where lacunae have inherent patterns and contexts.

Methods

The core contribution of this paper is the creation of the MAAT Corpus, which transforms existing ancient text data into a simplified, machine-actionable format. The process involves:

  • Data Conversion: Texts originally in EpiDoc XML format (a standard for epigraphy and papyrology) are converted into a simpler, Leiden-like format.
  • Salient Information Preservation: The conversion focuses on preserving only the most critical information for machine learning: the text itself, the location and extent of lacunae, and the textual restorations provided by human experts.
  • Machine-Actionable Format: The corpus is encoded using Unicode UTF-8, making it easily ingestible by machine learning systems.
  • Structured Test Cases: The corpus generates structured test cases that directly mirror the task of text restoration. This includes preserving the original text with gaps, the expert-provided restoration, and any alternative readings offered in the critical apparatus. This allows for evaluation metrics that consider the possibility of multiple correct restorations.
  • Handling of Lacunae: Different types of gaps (known length, approximate length, unknown length) are represented consistently. Gaps of known length are marked with dots corresponding to the number of missing letters, while unknown gaps use a specific tag.

Datasets

The MAAT Corpus is built upon existing large corpora of ancient texts, including:

  • DCLP (Digital Corpus of Literary Papyri in EpiDoc XML): Primarily Greek papyri.
  • DDbDP (Duke Databank of Documentary Papyri): A broad collection of papyrological texts.
  • EDH (Epigraphic Database Heidelberg): A significant collection of Latin inscriptions.

The initial 1.0 beta release of the MAAT Corpus contains approximately 134,000 text editions, 178,000 text blocks, and 750,000 individual text restorations. Greek and Latin texts dominate the corpus.

Key Insights and Ideas

  • Bridging the Gap between Human Expertise and AI: The MAAT Corpus directly addresses the need for data that aligns with the actual work of papyrologists and epigraphists, moving beyond purely synthetic evaluation tasks.
  • The Importance of Real-World Lacunae: Lacunae in ancient texts are not random; they follow a logarithmic distribution in length and occur in specific contexts. Evaluating models on these real lacunae provides a more accurate assessment of their performance.
  • Handling Ambiguity in Restoration: The corpus explicitly includes alternative readings from the critical apparatus, acknowledging that text restoration can often have multiple valid solutions. This is crucial for developing robust evaluation metrics that go beyond simple character accuracy.
  • Simplification for Actionability: The decision to strip away rich metadata and complex structural annotations from original formats like EpiDoc XML is key to making the data truly "machine-actionable" for LLMs.

Key Results

The paper introduces the MAAT Corpus as a resource, rather than presenting quantitative results from training models on it. However, it highlights the scale of the corpus:

  • Approximately 134,000 editions processed.
  • Approximately 178,000 ab blocks.
  • Approximately 750,000 individual text restorations.
  • Greek texts constitute about 53% of blocks and 83% of restorations, while Latin texts make up 46% of blocks and 16% of restorations.
  • Coptic texts represent a small fraction (around 1%).
  • The distribution of gap lengths in restored texts follows a logarithmic pattern, with 30% of gaps being a single character and 67% being four characters or less.

Limitations and Future Work

  • Beta Release: The current release is a 1.0 beta version, implying ongoing development and potential for refinement.
  • Limited Coptic Representation: The corpus has a small representation of Coptic texts.
  • Focus on Greek and Latin: While the goal is to expand, the current focus is heavily on Greek and Latin.
  • Future Work: The authors plan to:
  • Expand the corpus to include other ancient languages.
  • Develop pathways for converting data from other DSL-based formats into the MAAT Corpus.
  • Facilitate collaboration with other scholars and institutions to enrich the corpus.
  • Continue to improve the machine-actionable format and evaluation methodologies.

Summary updated

BibTeX
@inproceedings{fitzgerald2024new,
  author = {Will Fitzgerald and Justin Barney},
  title = {{A new machine-actionable corpus for ancient text restoration}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure