Beyond Static Leaderboards: Predictive Validity for the Evaluation of LLM Agents

Dhaval Patel, Kaoutar El Maghraoui, Shuxin Lin, Yusheng Li, Tianjun Feng, Chun-Yi Tsai, Yihan Sun, Wei Alexander Xin, Akshat Bhandari, Tanisha Rathod, Aaron Fan, Sanskruti Vijay Shejwal, Tomas Pasiecznik, Sagar Chethan Kumar, Tanmay Agarwal, Rohith Kanathur, Sam Colman, Amaan Sheikh, Dev Bahl, Ann Li, Krish Veera, Alimurtaza Mustafa Merchant, Shambhawi Bhure, Sajal Kumar Goyla, Chengrui Li, Kirthana Natarajan, Rui Li, Thomas Ajai, Rujing Li, Vivek G. Iyer, Sanjaii Vijayakumar, Yitong Bai, Ayal Yakobe, Darief Maes, Yassine Jebbouri, Tianyang Xu, Thai Quoc On, Vera Mazeeva, Winston Li, Yuval Shemla, Yeshitha Bhuvanesh, Rushin Bhatt, Siddharth Chethan Gowda, Alisha Vinod, Caroline Cahill, Shriya Aishani Rachakonda, Yunfeng Chen, Aryaman Agrawal, Aman Upganlawar, Mao Le Jonathan Ang, Yubin Sally Go, Madhav Rajkondawar, Yang-Jung Chen, Trisha Maturi, Ananya Kapoor, Andrew Li, Shrey Arora, Mana Abbaszadeh, Shen Li, Charles Xu, Byeolah KwonarXiv 2026 · DOI

BibTeX
@misc{patel2026beyond,
  author = {Dhaval Patel and Kaoutar El Maghraoui and Shuxin Lin and Yusheng Li and Tianjun Feng and Chun-Yi Tsai and Yihan Sun and Wei Alexander Xin and Akshat Bhandari and Tanisha Rathod and Aaron Fan and Sanskruti Vijay Shejwal and Tomas Pasiecznik and Sagar Chethan Kumar and Tanmay Agarwal and Rohith Kanathur and Sam Colman and Amaan Sheikh and Dev Bahl and Ann Li and Krish Veera and Alimurtaza Mustafa Merchant and Shambhawi Bhure and Sajal Kumar Goyla and Chengrui Li and Kirthana Natarajan and Rui Li and Thomas Ajai and Rujing Li and Vivek G. Iyer and Sanjaii Vijayakumar and Yitong Bai and Ayal Yakobe and Darief Maes and Yassine Jebbouri and Tianyang Xu and Thai Quoc On and Vera Mazeeva and Winston Li and Yuval Shemla and Yeshitha Bhuvanesh and Rushin Bhatt and Siddharth Chethan Gowda and Alisha Vinod and Caroline Cahill and Shriya Aishani Rachakonda and Yunfeng Chen and Aryaman Agrawal and Aman Upganlawar and Mao Le Jonathan Ang and Yubin Sally Go and Madhav Rajkondawar and Yang-Jung Chen and Trisha Maturi and Ananya Kapoor and Andrew Li and Shrey Arora and Mana Abbaszadeh and Shen Li and Charles Xu and Byeolah Kwon},
  title = {{Beyond Static Leaderboards: Predictive Validity for the Evaluation of LLM Agents}},
  year = {2026},
  eprint = {2606.19704},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.19704},
}

Loading related content...

Figure