Using YAGO for the Humanities

Thomas Rebele, Arash Nekoei, Fabian M. Suchanek

Abstract

The field of Digital Humanities relies on large-scale datasets to analyze historical trends, yet existing Knowledge Bases (KBs) like YAGO often suffer from incomplete data. This paper details a methodology for improving data completeness in YAGO by refining information extraction from Wikipedia. By tripling residence data and increasing gender data by 35%, the authors created a dataset of over 2.28 million individuals. This enhanced KB allows for statistically significant longitudinal studies, such as tracing life expectancy over 1,000 years and debunking the "full moon birth spike" myth.

Introduction

Digital Humanities applies computational methods to historical and cultural records. While the Semantic Web offers massive KBs like YAGO and DBpedia, these resources are frequently incomplete. For instance, early YAGO versions lacked gender information for nearly 40% of entries, and residence data was even more sparse. Traditional historical studies often rely on small, hand-curated datasets (150,000–300,000 people). To achieve global or millennial-scale analysis, researchers require the millions of entries found in KBs, provided the data is both complete and precise. This research focuses on "filling in the blanks" of YAGO without compromising accuracy.

Methods

The authors developed several specialized extractors to pull structured data from Wikipedia’s unstructured content:

  • Gender Extraction: The GenderCategoryExtractor scans categories (e.g., "Female people from Alexandria"), while the GenderNameExtractor uses a statistical Wilson estimator to assign gender based on first names with 95% confidence.
  • Date Reconciliation: To handle varied date formats in infoboxes, a system cross-references these dates with Wikipedia’s year-based categories (e.g., "1935 births") to ensure consistency.
  • Location Inference: The CategoryLocationExtractor identifies demonyms (e.g., "Egyptian") within categories to infer residence when birth or death places are missing.

Results and Data Analysis

The extraction pipeline significantly increased the volume of available data while maintaining high precision.

Table 1: Comparison of extraction results between YAGO versions and DBpedia.

The improved YAGO dataset now contains approximately 2.28 million people. This scale enables several key historical insights:

Life Expectancy and the 19th Century Crossover

The data allows for the tracking of median age at death over a millennium. A notable finding is the "19th-century crossover": for most of recorded history in this dataset, men lived longer than women. However, in the 19th century, this trend reversed, with women consistently outliving men—a shift that is statistically robust due to the high data density of that era.

Figure 1: Median age at death for males and females from 100 AD to 1900 AD.

The "Elite" and Relative Age Bias

The authors acknowledge a "notability bias," as YAGO primarily represents individuals successful enough to merit a Wikipedia page. This is evidenced by the "Relative Age Effect" in birth months: individuals born in January are overrepresented, largely due to sports league cutoffs that favor children born earlier in the year, who then achieve greater professional success.

Myth-Busting

By analyzing 400 years of birth data (1600–2000), the researchers found that 16.9% of births occurred near a full moon—matching the exact statistical probability of those days. This effectively debunks the popular myth of a "full moon birth spike."

Discussion and Conclusion

While the enhanced YAGO provides a powerful tool for historians, limitations remain. The "notability bias" means the data reflects the "elite" rather than the general population. Additionally, location extraction can occasionally produce anachronisms (e.g., using modern country names for historical regions). Despite these challenges, the integration of these extraction methods into the core YAGO infrastructure provides a significantly improved foundation for future Semantic Web research in the humanities.

Summary updated

BibTeX
@inproceedings{rebele2017using,
  author = {Thomas Rebele and Arash Nekoei and Fabian M. Suchanek},
  title = {{Using YAGO for the Humanities}},
  booktitle = {Whise@iswc},
  year = {2017},
}
Figure