TL;DR
This paper introduces a novel approach to automatically identify recurring narrative patterns, or "type-scenes," in Biblical Hebrew and Greek texts. By framing this as an information retrieval task and leveraging cross-references for training, the authors demonstrate a promising method for uncovering thematic connections that can aid Biblical scholarship.
What Problem Does This Paper Address?
Scholars have long recognized recurring narrative patterns, or "type-scenes," in ancient literature, such as the "man meets woman by a well" motif. Identifying these patterns manually is time-consuming and subjective. While computational methods have been applied to narrative analysis, detecting these subtle, genre-agnostic patterns in long, complex texts like the Bible remains a challenge. This paper aims to bridge this gap by developing a computational method to automatically detect these type-scenes, thereby offering a new perspective on narrative analysis and aiding scholars in their research.
Methods
The core of the proposed method is an information retrieval (IR) approach. The authors treat type-scene extraction as a task where a given verse (the query) is used to retrieve thematically similar verses (candidates). To train their model, they utilize cross-references within the Bible, assuming that linked passages share thematic similarities.
They employ a Siamese network architecture, instantiated with pre-trained BERT-like transformer models. This network has separate encoders for queries and candidates. The key innovation lies in training these encoders using a contrastive loss function. This function aims to pull the representations of linked (query-candidate) passages closer together in latent space while pushing apart representations of unrelated passages. This process is designed to learn representations that capture narrative similarity.
The authors also explore several experimental variations:
- Context Window Size: They investigate how the number of surrounding verses (context) affects retrieval performance, finding an optimal window of 4 verses (2 before and 2 after the query verse).
- Masking: They compare masking the query vector versus keeping it unmasked, finding that unmasking significantly improves performance.
- Batch Size: An ablation study reveals that a smaller batch size (16) is optimal for their contrastive learning setup.
- Language Models: They compare models trained on ancient languages (Hebrew/Greek) versus their modern counterparts, finding minimal performance differences but noting that one ancient Greek model outperformed others, potentially due to its training data.
Datasets
The primary datasets used are:
- Biblical Manuscripts:
- Ancient Hebrew: The Leningrad Codex (digitized version of the Hebrew Bible, without the Apocrypha).
- Ancient Greek: An open-source Ancient Greek edition of the Bible, including the Hebrew Bible (Septuagint) and the Christian New Testament.
- Cross-References: A dataset of cross-references from openbible.info, originally crowdsourced. This data was filtered to include only pairs with at least 5 votes, resulting in 45,297 Ancient Greek pairs and 17,899 Ancient Hebrew pairs.
- Type-Scene Queries: A manually crafted set of queries designed to elicit specific type-scenes, informed by humanities scholarship. These queries were translated into Ancient Hebrew and Ancient Greek.
Key Insights and Ideas
- Cross-references as a training signal: The paper demonstrates that cross-references, which capture various forms of textual connection (direct reuse, thematic parallelism, common characters), can effectively train models to identify deeper narrative similarities like type-scenes.
- Type-scenes as an IR problem: Framing type-scene detection as an information retrieval task is a novel and effective way to leverage existing IR techniques and models.
- Ancient vs. Modern Language Models: Surprisingly, the performance differences between models trained on ancient and modern languages were minimal. This suggests that while modern language models have more data, ancient language models can still perform well, especially if their training data is relevant (e.g., Septuagint-like Greek).
- Importance of query formulation: Concrete queries (e.g., "man meets woman by a well") perform better than abstract ones (e.g., "initiatory trial"). This highlights the impact of precise language in eliciting relevant results.
- Masking's negative impact: Unmasking the query verse in the Siamese network significantly boosts performance, contrary to some previous IR approaches. This suggests that the query itself is a crucial piece of information for identifying thematic similarity in this context.
Key Results
- The proposed Siamese network, fine-tuned on Bible cross-references, shows improved recall@k metrics compared to base models.
- While recall scores are still low in absolute terms (indicating the difficulty of retrieving a single verse from tens of thousands), the models consistently retrieve relevant candidates within the top 10 results for many type-scene queries.
- Ablation studies showed that an optimal context window size of 4 verses and a batch size of 16 were crucial for performance.
- Queries involving concrete nouns (e.g., "man," "woman," "well") tended to yield better results than more abstract queries.
- The model successfully identified some type-scenes (e.g., "Woman at the Well," "Miraculous Birth," "Water in the Desert") but struggled with more abstract or linguistically challenging ones (e.g., "God Sends a Prophet," "Dying Hero").
Limitations and Future Work
- Negative Sampling: The current random negative sampling strategy might not be optimal; exploring other strategies could improve results.
- Verse-level vs. Scene-level: Analyzing individual verses is insufficient for capturing entire narrative scenes. Future work should explore larger narrative windows or methods to identify narrative breaks/sections.
- LLM-generated Queries: The Greek queries were generated by an LLM, which may have introduced limitations due to a lack of nuanced understanding compared to a human expert.
- Circularity and Bias: Crowd-sourced votes for cross-references might be influenced by English translations and translator biases, potentially reinforcing existing interpretations. Confirmation bias can also lead to perceived patterns where none exist.
- Linguistic Discrepancies: The absence of direct equivalents for modern concepts like "hero" in ancient languages poses a significant challenge for query formulation and retrieval.
Future work could involve:
- More sophisticated methods for defining narrative sections.
- Removing masking from the training method.
- Adapting the approach to other long-form literary genres beyond the Bible.
Summary updated