TL;DR
This paper introduces a novel annotated corpus for Ancient Hebrew, focusing on named-entity and coreference resolution within the Hebrew Bible. By adapting existing Universal Dependencies (UD) guidelines and developing a custom annotation tool, the authors provide a valuable resource for linguistic analysis and information retrieval of this foundational text.
What Problem Does This Paper Address?
Understanding the relationships between entities (people, places, objects) and how they are referred to throughout a text is crucial for deep linguistic analysis and applications like information retrieval. While significant progress has been made in annotating modern languages, Ancient Hebrew, a language with immense historical and cultural significance, has lagged behind. Specifically, there was a lack of comprehensive datasets for named-entity recognition (NER) and coreference resolution (CR) in Ancient Hebrew, making it difficult to perform sophisticated text analysis on the Hebrew Bible. This paper aims to bridge this gap by creating such a dataset.
Methods
The authors built upon the existing Universal Dependencies (UD) framework for Ancient Hebrew, specifically the UD_Ancient_Hebrew-PTNK treebank. They adapted the CorefUD standard for coreference annotation, which uses a cluster-based approach where mentions point to an entry in a separate list of entities. To define what constitutes a mention and what entity types to use, they drew from the guidelines of the Universal Dependencies English GUM treebank.
A key methodological contribution is the development of a custom, simple terminal interface for annotation. This tool displays mentions and their context, allowing annotators to label them with existing entities or create new ones. Entities are identified by a unique ID composed of the first letter of their type and a sequential number. The annotation process involved both automated pre-processing and manual correction to ensure accuracy.
Datasets
The primary dataset used is a portion of the UD_Ancient_Hebrew-PTNK treebank, specifically the test and development sets, and half of the training set, covering the first 40 chapters of Genesis. This forms the basis for the newly introduced annotated corpus. The authors also reference the GUM corpus guidelines as a source for their annotation scheme.
Key Insights and Ideas
- Bridging Ancient Languages and Modern NLP: The project demonstrates the feasibility of applying sophisticated NLP annotation schemes, typically developed for modern languages, to ancient texts like the Hebrew Bible.
- Entity Granularity: The annotation process sometimes leads to a finer granularity of entities than typically found in external ontologies. For example, it distinguishes between a person and their name, or individual animals being sacrificed.
- Challenges of Ancient Hebrew: The paper highlights specific challenges in applying existing guidelines to Ancient Hebrew, such as the ambiguity of certain grammatical constructions and the need for careful definition of entity types like "nation."
- The Power of Custom Tools: The development of a simple, tailored annotation interface proved effective in managing the annotation process and facilitating annotator decisions.
Key Results
The annotated corpus covers the first 40 chapters of Genesis, resulting in over 10,000 mentions referring to almost 1500 distinct entities. The most common entity type is "Person," accounting for about 35% of entities and 70% of mentions. The paper reports inter-annotator agreement statistics, with span selection agreement at 88.61% (93.07% after correction), coreference agreement at 87.76%, and entity type agreement at 82.31%.
Limitations and Future Work
The authors acknowledge that the annotation guidelines, particularly for the "nation" entity type, were sometimes unclear, leading to disagreements. They also note that the automated coreference resolution tool (Xrenner) sometimes overgenerates spans or misclassifies mentions, requiring manual correction and automated post-processing.
Future work includes expanding the corpus to cover the rest of the Hebrew Bible. They also plan to incorporate other types of annotations, such as relationships between entities (e.g., part-whole) and linking entity IDs to external knowledge bases like Wikipedia, which would significantly enhance the resource's utility for complex querying.
Summary updated