Co-Training for Topic Classification of Scholarly Data

Cornelia Caragea, Florin Adrian Bulgarov, Rada Mihalcea

Abstract

As the volume of scientific literature grows, automated topic classification becomes essential. However, traditional supervised learning is limited by the high cost of expert labeling. This paper proposes a semi-supervised co-training approach that utilizes two independent views of a scholarly article: its internal content (title and abstract) and its external citation contexts. Our results demonstrate that this method achieves superior performance, reaching an F1-score of 0.742 with only 30% of labeled data, outperforming fully supervised models using 100% of the data.

Introduction

The "label bottleneck" is a significant hurdle in organizing scientific literature. While supervised machine learning can automate classification, it typically requires massive amounts of expert-labeled data. This study investigates whether the vast amount of unlabeled data in citation networks can be leveraged to improve classification. Specifically, we focus on citation contexts—the short snippets of text surrounding a citation—which act as external "micro-summaries" of a paper's contribution.

Methodology

We employ a Co-Training framework, a semi-supervised algorithm requiring two conditionally independent views of the same data:

  1. View 1 (Content): The paper’s title and abstract.
  2. View 2 (Citations): The aggregated text of all citation contexts associated with the paper.

The algorithm starts with a small pool of labeled papers. Two separate Naive Bayes classifiers are trained on each view. These models then iteratively identify the unlabeled examples they are most confident about and add them to the training set of the other classifier, allowing the models to bootstrap off each other’s strengths.

Experimental Results

The study utilized a dataset of 3,186 labeled papers from CiteSeerX across six computer science fields (Agents, AI, IR, ML, HCI, and DB).

F1-Score performance of individual views versus the integrated Co-Training approach across different training percentages.

As illustrated in the figure above, the Co-Training approach consistently outperforms classifiers trained on individual views. A key insight is that citation contexts often provide more descriptive power than the authors' own abstracts, as they filter out "fluff" and focus on the core topic as perceived by the research community.

Furthermore, the co-training framework proved significantly more robust than other semi-supervised baselines, such as self-training.

Comparison of Co-Training performance against Self-Training baselines for both Content and Citation views.

When the initial labeled data set is small (around 10%), co-training shows a marked advantage over self-training methods, which tend to propagate their own errors more readily than the mutually reinforcing co-training process.

Discussion and Conclusion

Co-training offers a highly efficient path for scholarly topic classification, achieving better results with 70% less labeled data than traditional methods. While the current study uses a fixed 100-word window for citation contexts, future work could involve more sophisticated natural language processing to identify the exact scope of citations. Additionally, addressing the "cold start" problem for new, uncited papers remains a priority. By integrating active learning and more complex topic distributions, we aim to further refine the accuracy and granularity of scholarly classification systems.

Summary updated

Abstract

With the exponential growth of scholarly data during the past few years, effective methods for topic classification are greatly needed. Current approaches usually require large amounts of expensive labeled data in order to make accurate predictions. In this paper, we posit that, in addition to a research article’s textual content, its citation network also contains valuable information. We describe a co-training approach that uses the text and citation information of a research article as two different views to predict the topic of an article. We show that this method improves significantly over the individual classifiers, while also bringing a substantial reduction in the amount of labeled data required for training accurate classifiers.
BibTeX
@inproceedings{caragea2015co,
  author = {Cornelia Caragea and Florin Adrian Bulgarov and Rada Mihalcea},
  title = {{Co-Training for Topic Classification of Scholarly Data}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2015},
  doi = {10.18653/v1/d15-1283},
}
Figure