Early Retrieval Problem and Link Prediction Evaluation via the Area Under the Magnified ROC

Alessandro Muscoloni, Carlo Vittorio CannistraciPreprints.org 2022 · DOI

Abstract

Link prediction is an unbalanced early retrieval problem, whose goal is to prioritize a small cohort of positive links on top of a list largely populated by unlabelled links. Differently from binary classification, here the evaluation focuses on how the predictor prioritizes the positive class because, in practice, a negative class does not exist. Previous studies explained that AUC-ROC is not apt for unbalanced class problems and is misleading for early retrieval problems, therefore standard AUC-ROC is not appropriate for evaluation of link prediction. However, some scholars argue that an AUC-ROC like evaluation accounting for the relative positioning of the few positive links among the vastness of unlabelled links remains a valid concept to pursue. Here we propose the area under the magnified ROC (AUC-mROC), a new measure that adjusts the standard AUC-ROC to work also for unbalanced early retrieval problems such as link prediction.
BibTeX
@misc{muscoloni2022early,
  author = {Alessandro Muscoloni and Carlo Vittorio Cannistraci},
  title = {{Early Retrieval Problem and Link Prediction Evaluation via the Area Under the Magnified ROC}},
  year = {2022},
  doi = {10.20944/preprints202209.0277.v1},
  howpublished = {Preprints.org},
}

Loading related content...

Figure