Introduction
Cross-referencing—the process of linking related passages across a large corpus—is a cornerstone of deep reading and academic scholarship. However, creating these links manually is a monumental task. For instance, the cross-references in the LDS edition of the Bible required hundreds of volunteers working for seven years. This is fundamentally an $O(n^2)$ search problem: as the text grows, the number of potential connections increases quadratically. The goal of this research is to lower the "expertise barrier" by using machine learning to surface the most likely connections for human review.
Methodology: Fine-Grained Topic Modeling
To address the search space problem, the system utilizes fine-grained topic modeling to generate candidate links for human verification. Unlike traditional "coarse" models that use roughly 100 topics, this approach utilizes 3,000 topics for a corpus of approximately 30,000 verses. This granularity ensures that each topic corresponds to a manageable number of documents, mirroring the density of human-curated references.
The technical approach leverages the Anchor Word algorithm, treated as a non-negative matrix factorization problem. A key innovation is Tandem Anchoring, where topics are seeded by the harmonic mean of word vectors within a randomly selected verse. This prevents the algorithm from selecting obscure or rare terms as anchors. Once trained, verses are represented as $K$-dimensional vectors, and similarity is calculated using various metrics, with cosine distance proving to be the most effective.
Evaluation and Results
The models were evaluated against three benchmarks: the Treasury of Scripture Knowledge, Enhanced (TSKE), and two subsets of crowdsourced data from OpenBible.info. To test generalizability, the model was also applied to the works of Jane Austen and Plato.
As shown in the performance metrics, fine-grained models (Tandem and Gram-Schmidt) significantly outperform traditional coarse models. In the Jane Austen corpus, 13% of the top 300 suggestions were valid thematic links, while in Plato's works, accuracy reached nearly 40%. Crucially, the fine-grained model identified "non-obvious" thematic links that simple keyword matching missed, such as connecting the "sting of a scorpion" to the "sting of death."
Discussion and Economic Impact
The primary value of this system is economic. By shifting the task from "expert search" to "novice verification," the cost of producing a comprehensive reference resource drops from millions of dollars in labor to approximately $7,500.
While the model is highly effective, it faces limitations with complex metaphors and redundant, nearly identical verses. Future work will focus on integrating semantic embeddings to better capture metaphorical relationships and refining anchor selection strategies to improve the discovery of subtle thematic connections.
Summary updated