Impact of Entity Disambiguation Errors on Social Network Properties

Jana Diesner, Craig Evans, Jinseok Kim

Abstract

Entities in social networks may be subject to consolidation when they are inconsistently indexed, and subject to splitting when multiple entities share the same name. How much do errors or shortfalls in entity disambiguation distort network properties? We show empirically how network analysis results and derived implications can tremendously change depending solely on entity resolution techniques. We present a series of controlled experiments where we vary disambiguation accuracy to study error propagation and the robustness of common network metrics, topologies and key players. Our results suggest that for email data, not conducting deduplication can make organizational communication networks appear less coherent and integrated as well as bigger than they truly are. For co-publishing networks, improper merging can make a scientific sector seem more dense and cohesive than it really is. Disambiguation errors can also lead to the false detection of power law distributions of node degree, suggesting preferential attachment processes that might not apply.

Introduction

In social network analysis (SNA), a fundamental assumption is that each node represents a unique individual. However, real-world datasets are frequently plagued by entity ambiguity. This manifests in two primary ways: consolidation problems, where a single individual utilizes multiple identifiers (e.g., different email addresses), and splitting problems, where multiple individuals share a common identifier (e.g., researchers with the same name). While algorithmic solutions for entity disambiguation exist, their impact on downstream network metrics is often underestimated. This study quantifies the "propagation of error" from raw data processing to final network interpretations.

Methodology and Datasets

The researchers conducted controlled experiments using two large-scale longitudinal datasets representing different disambiguation challenges:

  1. Enron Email Corpus: Comprising approximately 520,000 emails, this dataset serves as a testbed for consolidation. The "raw" data (email addresses) was compared against a manually "scrubbed" version where aliases were mapped to unique individuals.
  2. MEDLINE Co-publishing Data: Featuring approximately 101,000 papers, this dataset illustrates splitting issues. Common heuristics, such as "Surname + First Initial," were compared against a high-accuracy "Authority" database using metadata like affiliations to distinguish between authors.

Standard network metrics—including density, clustering coefficients, and centrality—were calculated across various levels of disambiguation accuracy to observe value fluctuations.

Analysis of Disambiguation Effects

The study reveals that disambiguation errors do not merely "nudge" results; they can fundamentally alter the perceived topology of a network.

False Topologies and Power Laws

One of the most significant findings is the impact on node degree distributions. Poor disambiguation can create an artificial "straight line" on log-log plots, leading researchers to incorrectly conclude that a network follows a power law distribution. This suggests a "rich-get-richer" preferential attachment process that may not exist in the actual social structure.

Impact of disambiguation heuristics on node degree distributions.

The "Super-Author" and "Fragmented Organization" Illusions

In co-authorship networks, failing to split common names creates "artifact entities"—fake nodes that appear extraordinarily productive and central. This makes the scientific field appear more cohesive than it is. Conversely, in organizational email data, failing to merge aliases makes the company appear larger and more fragmented, potentially leading to false conclusions about poor internal communication and organizational culture.

Quantitative Results

The scale of distortion is substantial. In the Enron dataset, the transition from raw email addresses to a scrubbed dataset resulted in an 81% reduction in the total number of nodes (addresses), illustrating how much raw data can inflate network size.

Reduction in network components after entity disambiguation in the Enron dataset.

In the MEDLINE data, network density was 97% higher in the heuristic-based version compared to the high-accuracy version. Centrality measures, used to identify key influencers, shifted by over 300% in some cases. Notably, while identifying "key players" was relatively robust in the Enron data, there was zero overlap between the top five players in the MEDLINE algorithmic versus heuristic versions.

Conclusion

Entity disambiguation is a critical, yet often overlooked, step in social network research. Errors in this phase propagate through the analysis, leading to potentially invalid conclusions about organizational health, scientific productivity, and network growth patterns. Future research should focus on identifying "sensitive" nodes or edges where disambiguation errors have the most catastrophic impact on results, allowing for targeted manual vetting in large-scale datasets where full scrubbing is infeasible.

Summary updated

Abstract

Entities in social networks may be subject to consolidation when they are inconsistently indexed, and subject to splitting when multiple entities share the same name. How much do errors or shortfalls in entity disambiguation distort network properties? We show empirically how network analysis results and derived implications can tremendously change depending solely on entity resolution techniques. We present a series of controlled experiments where we vary disambiguation accuracy to study error propagation and the robustness of common network metrics, topologies and key players. Our results suggest that for email data, not conducting deduplication, e.g. when operating on the level of email addressed instead of individuals, can make organizational communication networks appear to be less coherent and integrated as well as bigger than they truly are. For co-publishing networks, improper merging as caused by the commonly used initial based disambiguation techniques can make a scientific sector seem more dense and cohesive than it really is, and individual authors appear to be more productive, collaborative and diversified than they actually are. Disambiguation errors can also lead to the false detection of power law distributions of node degree; suggesting preferential attachment processes that might not apply.
BibTeX
@inproceedings{diesner2021impact,
  author = {Jana Diesner and Craig Evans and Jinseok Kim},
  title = {{Impact of Entity Disambiguation Errors on Social Network Properties}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/icwsm.v9i1.14630},
}
Figure