TL;DR
This paper introduces two novel datasets for automatically identifying the time period and location settings of literary works. These datasets, comprising over 2,300 time-period-labeled and nearly 7,000 location-labeled works, aim to facilitate computational literary analysis and improve question-answering systems by providing structured data on narrative settings.
What Problem Does This Paper Address?
Understanding the setting of a story—its time and place—is crucial for comprehending characters, plot, and even historical context. While this is recognized in literary analysis and early education, computational approaches have largely overlooked it. Existing datasets are either too small, lack diversity, or focus on publication time rather than narrative setting. This gap hinders large-scale "Cultural Analytics" of literature and the development of sophisticated literary Q&A systems that can leverage setting information for deeper understanding and inference.
Methods
The authors constructed their datasets by leveraging multiple sources and employing a combination of automated scraping and manual review.
For time period settings, they primarily used Project Gutenberg's public domain literary works. They extracted potential time period labels from Library of Congress (LoC) subject classifications, Wikipedia categories, and SparkNotes fact sheets. A significant part of their methodology involved meticulous manual cleaning and disambiguation of these labels. This included resolving variations in author names, handling missing dates, and distinguishing between the work's writing period and its narrative setting. They also developed a process to filter out labels that indicated the writing period rather than the story's setting.
For location settings, they similarly utilized LoC subject headings and SparkNotes fact sheets. They also incorporated data from Simple Maps for world cities, the USA, and Great Britain to enrich their location information. A key challenge addressed was handling variations in state names (abbreviations, postal codes) by creating a lookup table. Their approach aimed for a part-whole classification, allowing detection of country/state and city, and ensuring location legitimacy.
Datasets
The paper introduces two primary datasets:
- Time Period Setting Dataset: Contains 2,302 works labeled with time period settings. These labels are derived from LoC, Wikipedia, and SparkNotes, with a focus on narrative time period.
- Location Setting Dataset: Contains 6,991 works labeled with location settings. This dataset is built from LoC, SparkNotes, and external mapping data.
These datasets are made available under a Creative Commons Attribution 4.0 license.
Key Insights and Ideas
- The "Epistemic Role of Fiction": The paper highlights how fictional narratives, even when known to be untrue, can shape our understanding of the real world. This is particularly relevant for historical fiction, where settings can influence perceptions of past events (e.g., "Uncle Tom's Cabin" and the Civil War).
- Data Scarcity is a Major Hurdle: Despite the importance of settings, creating comprehensive datasets is challenging due to the scarcity of explicit, structured information in literary texts and the difficulty in disambiguating labels from various sources.
- Manual Curation is Essential: Automated methods alone are insufficient. The authors emphasize the necessity of extensive manual review and cleaning to ensure the accuracy and utility of the derived labels, especially when dealing with historical figures, fictional characters, and ambiguous date ranges.
- LoC Classifications are Underexplored: The paper suggests that Library of Congress classifications, a rich source of metadata, have been largely underutilized in natural language processing tasks, presenting an opportunity for future research.
Key Results
- Time Period Classification Baseline: Using TF-IDF features and a 3-way split for classification (early, middle, late periods), both Random Forest and Support Vector models achieved an F1 score of 0.81.
- Location Dataset Coverage: The location dataset includes 6,962 works with LoC subjects and 75 from SparkNotes, with 46 works overlapping between the two.
- Location Baseline: A simple term occurrence metric showed that 34.5% of works had the setting location as the most often mentioned location, and 60.5% had the setting location (or a larger encompassing location) mentioned.
Limitations and Future Work
- Approximations and Coarse-Graining: Many time period and location labels are approximations or coarse-grained (e.g., centuries, general geolocation). This limits their utility for highly granular analysis.
- Geographic and Linguistic Bias: The dataset primarily consists of English-language works, with a tendency towards Western settings. This bias needs to be considered when applying the dataset.
- Future Directions: The authors suggest tracking more detailed location information (e.g., specific buildings, character houses) and exploring more robust part-whole classification for locations. They also propose investigating the utility of LoC classifications more broadly in NLP and developing more sophisticated metrics for handling temporal thresholds.
Summary updated