Multi-level computational methods for interdisciplinary research in the HathiTrust Digital Library

Jaimie Murdock, Colin Allen, Katy Börner, Robert Light, Simon McAlister, Andrew Ravenscroft, Robert Rose, Doori Rose, Jun Otsuka, David Bourget, John Lawrence, Chris ReedarXiv 2017 · DOI · arXiv

TL;DR

This paper introduces a "drill-down" computational workflow that bridges the gap between large-scale statistical analysis ("distant reading") and detailed manual interpretation ("close reading"). By applying hierarchical topic modeling across varying levels of granularity—from entire library collections down to individual sentences—the researchers successfully identified nuanced historical arguments about animal cognition that standard keyword searches completely missed.

What Problem Does This Paper Address?

Interdisciplinary researchers often struggle to find relevant materials in massive digital archives like the HathiTrust. Traditional keyword searches fail because language is slippery: the same word can mean different things in different fields (polysemy) or change meaning over decades (terminological shift). For example, "anthropomorphism" refers to human-like gods in theology but human-like traits in animals in psychology.

Furthermore, digital libraries are "messy." They often lack structural markup (like chapter or paragraph breaks) and are riddled with Optical Character Recognition (OCR) errors. The authors sought a robust way to navigate these millions of volumes to find specific, high-quality arguments without requiring a human to read thousands of irrelevant books.

Methods

The authors developed a six-stage process that sequentially reduces the size of the dataset while increasing the resolution of the analysis.

Figure 1: Schematic rendering of the six-step process that sequentially drills down from macroscopic “distant reading” to microscopic “close reading” before zooming back out to the macroscopic scale at the final step. The approximate orders of magnitude of the datasets either side of each processing step are shown below the icons as powers of 10 of book/fulltext- sized units, and grey bars representing the data are scaled logarithmically.
  1. Keyword Search: An initial Solr search narrowed the HathiTrust collection to a manageable starting corpus (HT1315).
  2. Document-Level Topic Modeling: They used Latent Dirichlet Allocation (LDA) to identify broad themes across these volumes, filtering out irrelevant contexts (like theology).
  3. Page-Level Topic Modeling: They "drilled down" by treating individual pages as documents, identifying 86 volumes (HT86) with high concentrations of relevant pages.
  4. Argument Extraction: A human analyst used the OVA+ tool to manually map 43 specific arguments from 108 selected pages.
  5. Sentence-Level Topic Modeling: They applied LDA to 17,544 sentences within a single focal volume to find conceptually related ideas that lacked direct keyword overlap.
  6. Science Mapping: Finally, they "zoomed out" by creating a classification crosswalk to project these books onto the UCSD Map of Science, which is typically used for journal data.

Datasets

  • HathiTrust Digital Library: A massive collection of over 14.7 million volumes.
  • HT1315: A subset of 1,315 volumes retrieved via keyword search.
  • HT86: A further refined subset of 86 volumes.
  • HT6: Six focal volumes selected for deep argument analysis, including Margaret Washburn’s The Animal Mind (1908).
  • UCSD Map of Science: Used as a basemap for disciplinary visualization.

Key Insights and Ideas

  • Granularity Matters: The "aha moment" is that topic modeling isn't just for broad trends. By changing the unit of analysis (book $\rightarrow$ page $\rightarrow$ sentence), the model becomes a precision instrument for finding specific arguments.
  • Beyond Keywords: The system found relevant passages even when they didn't share words with the query. For instance, it linked a query about "psychic qualities" to a sentence about "angry wasps" because they shared a latent semantic context.
  • Resurrecting Lost Voices: The method spotlighted Margaret Washburn, a pioneer in comparative psychology whose foundational work had largely been overshadowed by the later rise of behaviorism.
  • Taxonomic Surprises: The "drill-down" revealed that early 20th-century scientists were debating the consciousness of organisms as simple as amoebae—a finding that surprised even the domain experts on the research team and connects historical work to modern studies on "slime mold intelligence."

Key Results

  • Search Effectiveness: None of the six focal volumes identified by the drill-down process appeared in the top 10 results of the initial HathiTrust keyword search.
  • Data Reduction: The workflow successfully reduced a search space of 300,000+ volumes down to 108 highly relevant pages for manual study.
  • Disambiguation: The LDA model (at $k=60$ topics) successfully partitioned "anthropomorphism" into distinct theological, biological, and philosophical clusters, allowing researchers to ignore the "noise" of religious texts.

Limitations and Future Work

  • Subjectivity of "Good" Topics: The authors acknowledge that assessing topic model quality remains an open problem. Human judgment of what makes a topic "coherent" does not always correlate with quantitative metrics of model fit.
  • Mapping Mismatches: The crosswalk between Library of Congress classifications (for books) and the UCSD Map of Science (for journals) is imperfect, sometimes placing books in uninterpretable regions of the map.
  • Automation: While the discovery process is highly automated, the final argument extraction still requires significant manual effort. The authors hope these methods will eventually support fully semi-automatic identification of argument structures.

Summary updated

Abstract

We show how faceted search using a combination of traditional classification systems and mixed-membership topic models can go beyond keyword search to inform resource discovery, hypothesis formulation, and argument extraction for interdisciplinary research. Our test domain is the history and philosophy of scientific work on animal mind and cognition. The methods can be generalized to other research areas and ultimately support a system for semi-automatic identification of argument structures. We provide a case study for the application of the methods to the problem of identifying and extracting arguments about anthropomorphism during a critical period in the development of comparative psychology. We show how a combination of classification systems and mixed-membership models trained over large digital libraries can inform resource discovery in this domain. Through a novel approach of "drill-down" topic modeling---simultaneously reducing both the size of the corpus and the unit of analysis---we are able to reduce a large collection of fulltext volumes to a much smaller set of pages within six focal volumes containing arguments of interest to historians and philosophers of comparative psychology. The volumes identified in this way did not appear among the first ten results of the keyword search in the HathiTrust digital library and the pages bear the kind of "close reading" needed to generate original interpretations that is the heart of scholarly work in the humanities. Zooming back out, we provide a way to place the books onto a map of science originally constructed from very different data and for different purposes. The multilevel approach advances understanding of the intellectual and societal contexts in which writings are interpreted.
BibTeX
@misc{murdock2017multi,
  author = {Jaimie Murdock and Colin Allen and Katy Börner and Robert Light and Simon McAlister and Andrew Ravenscroft and Robert Rose and Doori Rose and Jun Otsuka and David Bourget and John Lawrence and Chris Reed},
  title = {{Multi-level computational methods for interdisciplinary research in the HathiTrust Digital Library}},
  year = {2017},
  eprint = {1702.01090},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1702.01090},
}

Loading related content...

Figure