Commonly used software tools produce conflicting and overly-optimistic AUPRC values

Wenyu Chen, Miao Chen, Zhenghao Zhang, Cathy Sin‐Hang Fung, Ran Wang, Yizhen Chen, Qian Yan, Lixin Cheng, Kevin Y. Yip, Stephen Kwok‐Wing Tsui, Qin Cao

Abstract

The precision-recall curve (PRC) and the area under it (AUPRC) are useful for quantifying classification performance. They are commonly used in situations with imbalanced classes, such as cancer diagnosis and cell type annotation. We evaluated 10 popular tools for plotting PRC and computing AUPRC, which were collectively used in >3,000 published studies. We found the AUPRC values computed by the tools rank classifiers differently and some tools produce overly-optimistic results.
BibTeX
@inproceedings{chen2024commonly,
  author = {Wenyu Chen and Miao Chen and Zhenghao Zhang and Cathy Sin‐Hang Fung and Ran Wang and Yizhen Chen and Qian Yan and Lixin Cheng and Kevin Y. Yip and Stephen Kwok‐Wing Tsui and Qin Cao},
  title = {{Commonly used software tools produce conflicting and overly-optimistic AUPRC values}},
  booktitle = {Genome},
  year = {2024},
  doi = {10.1186/s13059-024-03266-y},
}

Loading related content...

Figure