A dataset of mentorship in bioscience with semantic and demographic estimations

Qing Ke, Lizhen Liang, Ying Ding, Stephen V. David, Daniel E. AcuñaScientific 2022 · DOI

TL;DR

Researchers have unveiled "Mentorship," a massive new dataset mapping 743,176 mentorship relationships across nearly 739,000 scientists, primarily in the biosciences. By linking crowdsourced academic genealogies with publication graphs and deep-learning-based semantic embeddings, this dataset provides an unprecedented look at how scientific influence, research topics, and demographics evolve across academic generations.

What Problem Does This Paper Address?

Mentorship is the "dark matter" of scientific success—we know it’s there and that it’s vital, but it’s incredibly hard to measure. Traditionally, researchers have relied on two proxies: doctoral dissertation records (which are often siloed by field) or co-authorship (which misses early-career interactions or mentors who don't share a byline with their students).

Existing datasets like the Mathematics Genealogy Project are excellent but lack publication data, while others like ProQuest are paywalled and lack researcher disambiguation. This paper addresses the need for a large-scale, open-access, and multi-disciplinary dataset that connects the "who mentored whom" (genealogy) with the "what did they actually do" (publications and semantics) and the "who are they" (demographics).

Methods

The authors built the dataset through a sophisticated pipeline of data integration and machine learning:

  1. Data Fusion: They started with the Academic Family Tree (AFT), a crowdsourced genealogy site, and linked it to the Microsoft Academic Graph (MAG).
  2. Matching Strategies: To link AFT profiles to MAG author IDs, they used a two-pronged approach: matching by co-authorship (finding papers where both mentor and mentee appear) and matching by name-institution pairs for those who haven't published together yet.
  3. Semantic Embedding: To represent a researcher’s "intellectual footprint," they used two methods: traditional TF-IDF and SPECTER, a transformer-based language model specifically trained on scientific document citations.
  4. Demographic Inference: They employed BERT-based deep learning models to estimate gender (from first names) and race/ethnicity (from full names), training on massive datasets like the SSA name database and Wikipedia profiles.

Datasets

The project integrates several major sources:

  • Academic Family Tree (AFT): The source for mentorship links.
  • Microsoft Academic Graph (MAG): The source for publication metadata and citations.
  • ORCID: Used for high-confidence validation of researcher identities.
  • Training Sets for Demographics: Author-ity 2009, Social Security Administration (SSA) names, Wikipedia, and the Ethnea tool.

Key Insights and Ideas

  • Mentorship is More Than Co-authorship: The dataset captures relationships that co-authorship-only models miss, particularly in the early stages of a career or in fields where mentors don't always co-publish with trainees.
  • The "Semantic Apple" Doesn't Fall Far from the Tree: A key validation step showed that 98.4% of PhD mentees are semantically closer to their mentors than to a randomly selected researcher. This confirms that mentors significantly shape the intellectual direction of their students.
  • SPECTER vs. TF-IDF: The authors found that SPECTER embeddings (which account for citation context) are superior to simple word-frequency models (TF-IDF) for representing a scientist's research area.
  • Quantifying Disparities: By including demographic estimations, the dataset allows researchers to move beyond anecdotes and statistically analyze how gender and race influence mentorship access and career longevity.

Key Results

  • Scale: 743,176 mentorship pairs among 738,989 scientists.
  • Matching Success: They successfully matched 63.8% of AFT researchers to their publication records in MAG.
  • High Accuracy: Gender prediction achieved F-1 scores > 0.96 on validation sets. Race/ethnicity prediction also performed well, with F-1 scores generally > 0.90 for most categories.
  • Disciplinary Breadth: While strongest in neuroscience (135k+ researchers) and chemistry (104k+), the dataset covers 20+ major areas including engineering, physics, and social sciences.

Limitations and Future Work

  • Demographic Bias: The authors honestly note that their race/ethnicity model struggles with African American names (F-1 score of ~0.30 on a specific test set). This is because the training data was skewed toward African names from outside the U.S., highlighting the difficulty of inferring ethnicity from names alone.
  • Crowdsourcing Limitations: Because AFT is crowdsourced, the data may suffer from selection bias—famous or highly active researchers are more likely to be included than those who left academia early.
  • Field Imbalance: The dataset is still heavily weighted toward the biosciences, reflecting the origins of the Academic Family Tree project. Expanding the coverage of social sciences and humanities remains a future goal.

Summary updated

Abstract

Mentorship in science is crucial for topic choice, career decisions, and the success of mentees and mentors. Typically, researchers who study mentorship use article co-authorship and doctoral dissertation datasets. However, available datasets of this type focus on narrow selections of fields and miss out on early career and non-publication-related interactions. Here, we describe Mentorship, a crowdsourced dataset of 743176 mentorship relationships among 738989 scientists primarily in biosciences that avoids these shortcomings. Our dataset enriches the Academic Family Tree project by adding publication data from the Microsoft Academic Graph and "semantic" representations of research using deep learning content analysis. Because gender and race have become critical dimensions when analyzing mentorship and disparities in science, we also provide estimations of these factors. We perform extensive validations of the profile-publication matching, semantic content, and demographic inferences, which mostly cover neuroscience and biomedical sciences. We anticipate this dataset will spur the study of mentorship in science and deepen our understanding of its role in scientists' career outcomes.
BibTeX
@inproceedings{ke2022dataset,
  author = {Qing Ke and Lizhen Liang and Ying Ding and Stephen V. David and Daniel E. Acuña},
  title = {{A dataset of mentorship in bioscience with semantic and demographic estimations}},
  booktitle = {Scientific},
  year = {2022},
  doi = {10.1038/s41597-022-01578-x},
}
Figure