TL;DR
This paper introduces a "drill-down" computational workflow that bridges the gap between large-scale statistical analysis ("distant reading") and detailed manual interpretation ("close reading"). By applying hierarchical topic modeling across varying levels of granularity—from entire library collections down to individual sentences—the researchers successfully identified nuanced historical arguments about animal cognition that standard keyword searches completely missed.
What Problem Does This Paper Address?
Interdisciplinary researchers often struggle to find relevant materials in massive digital archives like the HathiTrust. Traditional keyword searches fail because language is slippery: the same word can mean different things in different fields (polysemy) or change meaning over decades (terminological shift). For example, "anthropomorphism" refers to human-like gods in theology but human-like traits in animals in psychology.
Furthermore, digital libraries are "messy." They often lack structural markup (like chapter or paragraph breaks) and are riddled with Optical Character Recognition (OCR) errors. The authors sought a robust way to navigate these millions of volumes to find specific, high-quality arguments without requiring a human to read thousands of irrelevant books.
Methods
The authors developed a six-stage process that sequentially reduces the size of the dataset while increasing the resolution of the analysis.
- Keyword Search: An initial Solr search narrowed the HathiTrust collection to a manageable starting corpus (HT1315).
- Document-Level Topic Modeling: They used Latent Dirichlet Allocation (LDA) to identify broad themes across these volumes, filtering out irrelevant contexts (like theology).
- Page-Level Topic Modeling: They "drilled down" by treating individual pages as documents, identifying 86 volumes (HT86) with high concentrations of relevant pages.
- Argument Extraction: A human analyst used the OVA+ tool to manually map 43 specific arguments from 108 selected pages.
- Sentence-Level Topic Modeling: They applied LDA to 17,544 sentences within a single focal volume to find conceptually related ideas that lacked direct keyword overlap.
- Science Mapping: Finally, they "zoomed out" by creating a classification crosswalk to project these books onto the UCSD Map of Science, which is typically used for journal data.
Datasets
- HathiTrust Digital Library: A massive collection of over 14.7 million volumes.
- HT1315: A subset of 1,315 volumes retrieved via keyword search.
- HT86: A further refined subset of 86 volumes.
- HT6: Six focal volumes selected for deep argument analysis, including Margaret Washburn’s The Animal Mind (1908).
- UCSD Map of Science: Used as a basemap for disciplinary visualization.
Key Insights and Ideas
- Granularity Matters: The "aha moment" is that topic modeling isn't just for broad trends. By changing the unit of analysis (book $\rightarrow$ page $\rightarrow$ sentence), the model becomes a precision instrument for finding specific arguments.
- Beyond Keywords: The system found relevant passages even when they didn't share words with the query. For instance, it linked a query about "psychic qualities" to a sentence about "angry wasps" because they shared a latent semantic context.
- Resurrecting Lost Voices: The method spotlighted Margaret Washburn, a pioneer in comparative psychology whose foundational work had largely been overshadowed by the later rise of behaviorism.
- Taxonomic Surprises: The "drill-down" revealed that early 20th-century scientists were debating the consciousness of organisms as simple as amoebae—a finding that surprised even the domain experts on the research team and connects historical work to modern studies on "slime mold intelligence."
Key Results
- Search Effectiveness: None of the six focal volumes identified by the drill-down process appeared in the top 10 results of the initial HathiTrust keyword search.
- Data Reduction: The workflow successfully reduced a search space of 300,000+ volumes down to 108 highly relevant pages for manual study.
- Disambiguation: The LDA model (at $k=60$ topics) successfully partitioned "anthropomorphism" into distinct theological, biological, and philosophical clusters, allowing researchers to ignore the "noise" of religious texts.
Limitations and Future Work
- Subjectivity of "Good" Topics: The authors acknowledge that assessing topic model quality remains an open problem. Human judgment of what makes a topic "coherent" does not always correlate with quantitative metrics of model fit.
- Mapping Mismatches: The crosswalk between Library of Congress classifications (for books) and the UCSD Map of Science (for journals) is imperfect, sometimes placing books in uninterpretable regions of the map.
- Automation: While the discovery process is highly automated, the final argument extraction still requires significant manual effort. The authors hope these methods will eventually support fully semi-automatic identification of argument structures.
Summary updated