TL;DR
This paper introduces a novel end-to-end framework for automatically identifying and disambiguating author affiliations from scientific literature. By combining word embeddings with spatial embeddings, the framework effectively handles the ambiguity and incompleteness common in affiliation data, paving the way for more accurate bibliometric analyses.
What Problem Does This Paper Address?
Bibliometrics, the quantitative analysis of scholarly literature, is crucial for understanding research trends and impact. A key component of this analysis is author affiliation information, which tells us where research is happening. However, this information is often messy: affiliations can be written in inconsistent ways (e.g., "Univ. of Cambridge" vs. "Cambridge University"), use abbreviations (e.g., "UCLA"), or be incomplete. This ambiguity makes it difficult for automated systems to accurately extract and link affiliations to specific institutions or locations, leading to inaccuracies in bibliometric studies. Existing methods often struggle with this, either focusing only on institutions and ignoring locations, or relying on local text features without considering broader context.
Methods
The proposed framework tackles this challenge in two main stages: preprocessing and disambiguation.
Preprocessing: 1. Affiliation Extraction: A rule-based approach using regular expressions is employed to identify and extract affiliation strings from PubMed data, specifically looking for the "AD –" prefix. 2. Entity Recognition: The GliNER model, a Transformer-based named entity recognition (NER) model, is used to identify "organization" and "location" entities within the extracted affiliation text. GliNER was chosen for its strong performance, especially in zero-shot scenarios, and its superiority over other NER tools on this type of data.
Disambiguation: This is the core of the paper's novelty. It involves two steps:
- Candidate Generation: For each identified entity (e.g., "University of California"), candidate entities are retrieved from the DBpedia knowledge graph. This process leverages DBpedia's rich information about entities, including attributes and relationships, to generate a comprehensive list of potential matches.
- Candidate Ranking: A novel algorithm combines two types of similarity scores to rank the candidate entities:
- Context Similarity: This uses pre-trained GloVe word embeddings to capture the semantic relationships between the entity mention and its surrounding text, comparing it to the semantic information of candidate entities.
- Geographic Similarity: This leverages spatial embeddings and the GeoNames database. It considers the geographical information present in the affiliation text and the candidate entity's abstract, assuming that locations mentioned frequently in a candidate's description are more likely to be associated with it. This is particularly useful for disambiguating locations.
The final disambiguation score is a weighted combination of these two similarity scores.
Datasets
The primary dataset used in this work is author affiliation data exported from the PubMed database. The paper mentions that approximately 9,000 records were manually annotated for training and evaluation. The dataset was divided into three subsets for K-fold cross-validation.
Key Insights and Ideas
- The Power of Combined Embeddings: The paper highlights that simply using word embeddings isn't enough. Incorporating spatial embeddings, which capture geographic relationships, significantly improves the disambiguation of location entities. This is a crucial insight for affiliation data, which is inherently tied to geography.
- GeoNames Limitations and Workarounds: While GeoNames is a valuable resource, the paper points out its limitations in handling country aliases (e.g., "UK"). This leads to a future work suggestion of creating a priority mapping table for such aliases.
- Stability Across Data Scales: The proposed model demonstrates robust performance across different training dataset sizes, indicating its stability and reliability even with limited training data.
- Geographic Information Boosts Precision: The ablation studies show that geographical information primarily boosts the precision of entity disambiguation, meaning it's more effective at correctly identifying the right entity rather than just finding any plausible entity.
Key Results
- Improved Performance: The proposed algorithm achieves strong performance, outperforming baseline methods like rule-based approaches and the WeLink model. For example, in Table 2, the proposed algorithm achieves an F1 score of 0.758, compared to 0.639 for rule-based and 0.722 for WeLink.
- Location Entity Disambiguation: Ablation experiments show a significant improvement for Location entities when geographic information is included. Precision increased by 0.288, Recall by 0.221, and F1 by 0.179.
- Organization Entity Disambiguation: Organization entities also saw improvements, with accuracy increasing by 13.5%, recall by 3.8%, and F1 by 8.1%.
- Dataset Size Stability: Table 1 shows minimal variation in Precision, Recall, and F1-score across small, medium, and large training datasets, indicating the model's stability.
Limitations and Future Work
- Limitations:
- The paper acknowledges that some affiliation data is severely lacking in correct and necessary information, which can still lead to incorrect results even with their improved algorithm.
- The current focus is primarily on "Location" and "Organization" entity types.
- Future Work:
- Extend the disambiguation algorithm to analyze other entity types, such as author names.
- Utilize the Research Organization Registry (ROR) as a knowledge repository to further enhance the system.
- Address the limitations of GeoNames by creating a higher priority mapping table for country aliases.
Summary updated