QiMeng-LibBench: Benchmarking LLM Agents for Library-Scale Cross-Architecture Migration
Lists
Only you see this
Abstract
Cross-architecture migration of high-performance libraries dictates ecosystem readiness on emerging hardware. The challenge is twofold: disentangling library-scale dependencies and performance-critical kernels with ISA-specific SIMD intrinsics, often trading migration speed for peak performance. While LLM-based agents offer a promising approach, are confined to function-level tasks or scalar code, failing to assess agents’ capabilities and limitations in realistic, library-scale migration. We present QM-LibBench, a benchmark for cross-architecture library-scale code migration, featuring 85 critical kernels from widely used libraries, including OpenCV, libjpeg, and NCNN. It supports comprehensive evaluations of compilability, correctness, and performance across major transitions: ARM→RISC-V, x86→ARM, and ARM→LoongArch. Evaluation of 12 SOTA agent-LLM combinations on QM-LibBench reveals that, due to the lack of library-level navigation and hardware-aware optimization, agents regress to superficial pattern matching, yielding only 20.88\% correctness and 0.83 speedup for libjpeg. Motivated by these findings, we further propose FSCM, a multi-agent framework incorporating hardware-aware global reconfiguration and performance optimization. FSCM improves OpenCV correctness to 71\%. The benchmark and code are available at https://github.com/WisdomJoy/QM-LibBench.
BibTeX
@inproceedings{li2026qimeng,
author = {Weijia Li and KE GAO and Jiajie Li and Han Sun and Yuhe Ding and Yongdong Mai and Yiran Le and Yongjie Qian and Zhibin Zhang and Xinyu Wang and Limin Cheng and Shouxu Kuang and Pengfei Chen and Ling Li},
title = {{QiMeng-LibBench: Benchmarking LLM Agents for Library-Scale Cross-Architecture Migration}},
booktitle = {International Conference on Machine Learning},
year = {2026},
url = {https://openreview.net/forum?id=uhSqCIVc0B},
}Loading related content...