Early Retrieval Problem and Link Prediction Evaluation via the Area Under the Magnified ROC
Lists
Only you see this
Abstract
Link prediction is an unbalanced early retrieval problem, whose goal is to prioritize a small cohort of positive links on top of a list largely populated by unlabelled links. Differently from binary classification, here the evaluation focuses on how the predictor prioritizes the positive class because, in practice, a negative class does not exist. Previous studies explained that AUC-ROC is not apt for unbalanced class problems and is misleading for early retrieval problems, therefore standard AUC-ROC is not appropriate for evaluation of link prediction. However, some scholars argue that an AUC-ROC like evaluation accounting for the relative positioning of the few positive links among the vastness of unlabelled links remains a valid concept to pursue. Here we propose the area under the magnified ROC (AUC-mROC), a new measure that adjusts the standard AUC-ROC to work also for unbalanced early retrieval problems such as link prediction.
BibTeX
@misc{muscoloni2022early,
author = {Alessandro Muscoloni and Carlo Vittorio Cannistraci},
title = {{Early Retrieval Problem and Link Prediction Evaluation via the Area Under the Magnified ROC}},
year = {2022},
doi = {10.20944/preprints202209.0277.v1},
howpublished = {Preprints.org},
}Loading related content...