Bayesian Non-Exhaustive Classification for Active Online Name Disambiguation

Baichuan Zhang, Murat Dundar, Mohammad Al HasanarXiv 2017 · OpenReview

TL;DR

This paper introduces a novel Bayesian framework for "online name disambiguation," a challenging task that identifies unique individuals from a stream of records associated with common names. Unlike previous methods that assume all individuals are known upfront, this approach can also discover new individuals as new records arrive, making it ideal for dynamic, real-world data.

What Problem Does This Paper Address?

In many real-world scenarios, such as academic publications or social media, common names are shared by numerous individuals. The "name disambiguation" task aims to correctly group records (e.g., papers, posts) to their respective owners. Traditional methods often assume a static dataset where all individuals are known beforehand. However, this assumption breaks down in streaming data environments where new individuals, and thus new "classes" of records, can emerge over time. This "non-exhaustive" nature of the data poses a significant challenge. Existing online methods often rely on heuristic thresholds, making them sensitive to parameter choices and less robust. The authors address the need for a flexible, adaptive system that can handle both known and unknown individuals in a continuous data stream, while also being interactive to leverage user feedback.

Methods

The core of the proposed solution is a Bayesian Non-Exhaustive Classification framework built upon a Dirichlet Process Gaussian Mixture Model (DPGMM). The DPGMM is particularly well-suited for this problem because it can model an unknown number of clusters (individuals) and adapt as new data arrives.

To handle the streaming nature of the data, two online inference algorithms are proposed:

  • One-Pass Gibbs Sampler: This algorithm processes each incoming record sequentially, deciding whether it belongs to an existing individual or a new one. It uses a conditional posterior probability derived from the DPGMM and a Chinese Restaurant Process (CRP) prior to make these decisions.
  • Particle Filtering (Sequential Importance Sampling with Resampling): This method offers a more robust approach by maintaining a set of "particles," each representing a possible configuration of individual assignments. As new records arrive, particle weights are updated, and resampling is used to prevent particle degeneracy. This allows for a more comprehensive exploration of the state space and better handling of uncertainty.

Additionally, the authors introduce an active learning component. This allows the system to identify records where its disambiguation confidence is low and request user feedback, thereby improving its accuracy and adapting more effectively.

Datasets

The primary dataset used for evaluation is the Arnetminer bibliographic dataset. This dataset is chosen because it provides a temporal stream of academic publications, a common scenario where name disambiguation is crucial. The authors specifically selected 25 highly ambiguous name references from Arnetminer, providing detailed statistics on the number of records, attributes, and distinct authors for each. This dataset is used to simulate real-world streaming scenarios.

Key Insights and Ideas

  • The "Non-Exhaustive" Challenge is Crucial: The paper strongly emphasizes that real-world data streams are rarely exhaustive. Algorithms that assume all possible individuals are known upfront will inevitably fail when new individuals appear.
  • DPGMM as a Natural Fit: The Dirichlet Process is a powerful tool for modeling an unknown number of components, making it ideal for discovering new individuals (classes) in a streaming context.
  • Particle Filtering Outperforms Gibbs Sampling: While both inference methods work, particle filtering demonstrates superior performance, likely due to its ability to maintain multiple hypotheses and avoid error propagation more effectively than the single-hypothesis Gibbs sampler.
  • Active Learning Significantly Boosts Performance: Incorporating user feedback, especially for uncertain disambiguations, dramatically improves accuracy, highlighting the value of human-in-the-loop systems.
  • Feature Importance: The paper shows that combining co-author information, keywords, and publication venue provides the best performance, suggesting a multi-faceted approach to disambiguation is most effective.

Key Results

  • The proposed Bayesian non-exhaustive classification framework, particularly with the particle filtering inference, significantly outperforms existing online name disambiguation methods (Qian's method, Khabsa's method) and offline baselines (BernouNaive-HAC, NNMF-SVM-HAC) on the Arnetminer dataset.
  • For instance, on the "Jing Zhang" name reference, the particle filtering approach achieved a mean-F1 score of 0.601, a substantial improvement over the best competing method (BernouNaive-HAC) which scored 0.480 (a 25.2% improvement).
  • The active learning component, when compared to random selection, consistently achieved higher mean-F1 scores for the same ratio of queried records, demonstrating the efficiency of its "active selection" strategy.
  • The model's running time scales linearly with the number of records, making it efficient for large datasets.

Limitations and Future Work

  • Data Model Simplicity: The current model primarily uses Gaussian distributions for class components. The authors suggest extending this to more flexible class distributions or hierarchical Dirichlet Process priors for improved modeling.
  • Temporal Dynamics: While the data is processed chronologically, the authors propose incorporating more explicit temporal dynamics into the prior model to better capture trends like author career progression.
  • Feature Engineering: The current feature set is limited to co-authors, keywords, and venue. Future work could explore richer feature representations.
  • Active Learning Threshold: The effectiveness of the active learning strategy depends on the entropy-based threshold. Further research could explore more sophisticated criteria for query selection.

Summary updated

Abstract

The name disambiguation task partitions a collection of records pertaining to a given name, such that there is a one-to-one correspondence between the partitions and a group of people, all sharing that given name. Most existing solutions for this task are proposed for static data. However, more realistic scenarios stipulate emergence of records in a streaming fashion where records may belong to known as well as unknown persons all sharing the same name. This requires a flexible name disambiguation algorithm that can not only classify records of known persons represented in the train- ing data by their existing records but can also identify records of new ambiguous persons with no existing records included in the initial training dataset. Toward achieving this objective, in this paper we propose a Bayesian non-exhaustive classification frame- work for solving online name disambiguation. In particular, we present a Dirichlet Process Gaussian Mixture Model (DPGMM) as a core engine for online name disambiguation task. Meanwhile, two online inference algorithms, namely one-pass Gibbs sampler and Sequential Importance Sampling with Resampling (also known as particle filtering), are proposed to simultaneously perform online classification and new class discovery. As a case study we consider bibliographic data in a temporal stream format and disambiguate authors by partitioning their papers into homogeneous groups.Our experimental results demonstrate that the proposed method is significantly better than existing methods for performing online name disambiguation task. We also propose an interactive version of our online name disambiguation method designed to leverage user feedback to improve prediction accuracy.
BibTeX
@misc{zhang2017bayesian,
  author = {Baichuan Zhang and Murat Dundar and Mohammad Al Hasan},
  title = {{Bayesian Non-Exhaustive Classification for Active Online Name Disambiguation}},
  year = {2017},
  eprint = {1708.04531},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1708.04531},
}
Figure