League: Leaderboard Generation on Demand

Jian Wu, Jiayu Zhang, Luodan Zhang, Dongyuan Li, Linyi Yang, Aoxiao Zhong, Renhe Jiang, Qingsong Wen, Yue ZhangSubmission to ICLR 2026 · Withdrawn

Its authors withdrew this paper, so there is no full text to read or summarize.

Abstract

Leaderboard gathers experimental results from various sources into unified rankings, giving researchers clear standards for measuring progress while facilitating fair comparisons. However, with thousands of academic papers updated daily, manually tracking each paper's methods, results, and experimental settings has become impractical, creating an urgent need for automated leaderboard generation. Although large language models offer promise in automating this process, challenges such as multi-document summarization, fair result extraction, and consistent experimental comparison remain underexplored. To address these challenges, we introduce Leaderboard Auto Generation (League), a novel and well-organized framework for automatic generation of leaderboards on a given research topic in rapidly evolving fields like Artificial Intelligence. League employs a systematic pipeline encompassing paper collection, result extraction and integration, leaderboard generation, and quality evaluation. Through extensive experiments across multiple research domains, we demonstrate that League produces leaderboards comparable to manual curation while significantly reducing human effort.
BibTeX
@misc{wu2025league,
  author = {Jian Wu and Jiayu Zhang and Luodan Zhang and Dongyuan Li and Linyi Yang and Aoxiao Zhong and Renhe Jiang and Qingsong Wen and Yue Zhang},
  title = {{League: Leaderboard Generation on Demand}},
  year = {2025},
  eprint = {2502.18209},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.18209},
}
Figure