AscendGYM: A Cost-Aware Kernel Optimization Benchmark for Real-World Workloads on Ascend NPUs

Ai Training Platform Team, Bingwei Zhang, Jing Li, Amir Abdullin, Sihang Chen, Xin Chen, Liangjun Feng, Yingmeng Gao, Tao Guo, Junjie Huang, Dongfang Li, Zhanglian Lin, Jiajun Liu, Zekun Liu, Zhengxuan Lu, Xiaodong Luo, Jian Meng, Ivan Nosov, Linyuan Qiu, Weijian Sun, Wanrui Yan, Binfan Zheng, Zhicheng Zhou, Baotian Hu, Min Zhang, Haizhou Li, Zhiquan LuoarXiv 2026 · arXiv

TL;DR

AscendGYM is a new benchmark designed to evaluate how effectively AI agents can optimize existing high-performance kernels for Ascend NPUs using real-world LLM workloads. It introduces the Speedup–Token Score (STS), a novel metric that balances performance gains against the computational cost (tokens) required to achieve them, ensuring that optimizations are both effective and economically viable.

What Problem Does This Paper Address?

Efficient Large Language Model (LLM) training and inference rely heavily on highly optimized "kernels"—the fundamental mathematical operations (like matrix multiplication or normalization) that run on specialized hardware like Ascend NPUs.

While several benchmarks exist for generating kernels from scratch, they often fall short in three areas:

  1. Optimization vs. Generation: In the real world, engineers usually start with a working kernel and try to make it faster. Most benchmarks don't test this iterative optimization process.
  2. Synthetic vs. Real Workloads: Performance varies wildly based on tensor shapes and data types. Many benchmarks use synthetic configurations that don't reflect the actual bottlenecks found in production LLM pipelines.
  3. The Hidden Cost of Search: Finding a faster kernel can be incredibly expensive in terms of LLM tokens. A 10% speedup that costs $1,000 in API fees might not be worth it. Existing metrics ignore this trade-off.

Methods

The authors developed AscendGYM, a framework that provides a realistic "gym" for AI agents to practice kernel optimization.

The AscendForge Sandbox

To make evaluation reliable, the authors built AscendForge, an execution sandbox that handles the complex plumbing of NPU development. It automates compilation, correctness checking (comparing outputs against a "golden" CPU reference), and precise performance profiling.

Figure 1: Architecture of AscendForge, integrating resource orchestration, kernel execution, evalua- tion and feedback, and deployment management for Ascend kernel optimization.

The Speedup–Token Score (STS)

The standout contribution is the STS metric. It rewards the final speedup achieved but applies a penalty for "delayed improvements." If an agent finds a massive speedup in its first few attempts, it scores higher than an agent that finds the same speedup after thousands of iterations. This encourages "budget-aware" optimization.

Evaluation Workflow

The workflow mimics a human engineer's loop: read the spec, run a baseline, analyze bottlenecks, propose code changes, and verify results in the sandbox.

Figure 2: Overview of the AscendGYM workflow.

Datasets

AscendGYM introduces a suite of 48 kernel optimization tasks categorized by difficulty:

  • Batch 1 (17 kernels): Focuses on memory access patterns in basic kernels (e.g., RMSNorm, SwiGLU).
  • Batch 2 (17 kernels): Covers multi-step computations like quantization.
  • Batch 3 (14 kernels): Targets complex attention mechanisms and fused operators.

Crucially, the tensor shapes and data types for these kernels are extracted directly from real-world models like Qwen3-0.6B and frameworks like vLLM-Ascend, ensuring that improvements in the benchmark translate to real-world speedups.

Key Insights and Ideas

  • Speedup is not enough: The authors found that two systems might reach the same final speedup but differ vastly in efficiency. One might find the gain in 5 attempts, while another takes 50. STS successfully captures this distinction.
  • The "Warm Start" Advantage: Using a "warm" configuration—where the agent is provided with "lessons" or strategies learned from optimizing other kernels—significantly improves the STS. This suggests that cross-task experience is vital for efficient optimization.
  • Correctness $\neq$ Performance: High "Pass Rates" (producing code that works) do not guarantee high speedups. For example, GPT-5.6 passed nearly all tasks but achieved significantly lower speedups than Claude Opus 5 on certain harnesses.
  • Real-world difficulty: Even for SOTA models, optimizing existing kernels is hard. Approximately 70% of evaluated model-harness combinations achieved a geometric mean speedup of less than 1.2x.

Key Results

The researchers evaluated several leading models (Claude Opus 5, GPT-5.6, GLM-5.3, Qwen series) across different agent harnesses (Claude Code, Codex, OpenCode).

  • Top Performer: Claude Opus 5 paired with the Claude Code harness generally achieved the highest performance gains, though often at a higher token cost.
  • Efficiency: Under a "tight" budget of 10M tokens, the "Warm" default harness configuration outperformed "Cold" starts significantly, proving that accumulated optimization strategies are a force multiplier.
  • Task Coverage: While some models could accelerate all tasks in Batch 1, only a few could achieve speedups greater than 2x on more than a handful of kernels (e.g., fused_gdn_gating saw a 3.29x improvement).

Limitations and Future Work

  • Scope: This initial report focuses primarily on the 17 kernels in Batch 1. Results for the more complex Batch 2 and 3 kernels are pending.
  • Hardware Specificity: The benchmark is currently tailored to Ascend 910C NPUs. While the methodology is generalizable, the specific kernels and optimizations are hardware-dependent.
  • Future Directions: The authors plan to expand the task suite to include more diverse workloads and explore graph-level optimizations (fusing multiple kernels together) rather than just single-kernel tuning.

Summary updated

Abstract

Kernel optimization is essential for efficient large language model (LLM) training and inference on Ascend NPUs. Existing Ascend kernel benchmarks primarily focus on kernel generation, with less emphasis on the ability of agents to optimize existing implementations under input configurations drawn from real workloads. To address this requirement, we introduce AscendGYM, a benchmark whose initial suite comprises 48 kernels across three difficulty levels. Each kernel is paired with test cases whose data types and tensor shapes are collected from real-world training and inference tasks. Consequently, performance improvements on AscendGYM map directly to the corresponding workloads. In terms of usability, two evaluation modes are supported in AscendGYM: model evaluation and model-harness evaluation. The kernel optimization capabilities of both trained models and custom agents are allowed to be measured. It provides a complete execution environment that closely reproduces how Ascend kernels are invoked in practical pipelines. For model evaluation, it additionally includes a built-in harness equipped with search orchestration and experience accumulation. Finally, we propose a new metric, the Speedup-Token Score (STS), which quantifies performance gains relative to token cost. By jointly considering speedup and cost, STS enables the assessment of practical kernel optimization capability of a model or harness under an acceptable budget. AscendGYM is available at https://github.com/SLAI-AITP/AscendGYM.
BibTeX
@misc{team2026ascendgym,
  author = {AI Training Platform Team and Bingwei Zhang and Jing Li and Amir Abdullin and Sihang Chen and Xin Chen and Liangjun Feng and Yingmeng Gao and Tao Guo and Junjie Huang and Dongfang Li and Zhanglian Lin and Jiajun Liu and Zekun Liu and Zhengxuan Lu and Xiaodong Luo and Jian Meng and Ivan Nosov and Linyuan Qiu and Weijian Sun and Wanrui Yan and Binfan Zheng and Zhicheng Zhou and Baotian Hu and Min Zhang and Haizhou Li and Zhiquan Luo},
  title = {{AscendGYM: A Cost-Aware Kernel Optimization Benchmark for Real-World Workloads on Ascend NPUs}},
  year = {2026},
  eprint = {2611.17621},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17621},
}
Figure