When a language model rewrites a GPU kernel over many rounds of compiling, timing, and fixing, the final code mixes two kinds of change. Some changes are real algorithmic improvements: skipping a redundant computation, fusing operations so intermediate results never leave the chip. Others are bets on one particular chip: a tile size matched to its cache, a block-scheduling pattern matched to its die layout, an instruction that only it has. Current refinement agents keep the edits that make the clock go down on the machine in front of them, and they never record which kind each edit was. The result reaches 1.68× over PyTorch on an NVIDIA GPU, but nothing in it says how much of that gain will carry to an AMD or Intel part, or even to the next NVIDIA generation.
We propose to make that distinction explicit and measurable. The refinement loop keeps each accepted edit as a separate, replayable patch in a ledger. After optimization, every edit is scored by how much speed it contributes on the source device and on a target device. Porting then keeps the edits that transfer, strips the ones that hurt, and spends new search budget only where the ledger shows a device-specific hole. The core claim has two parts. First, as refinement goes deeper, gains become steadily more device-specific, and denser profiler feedback speeds this drift up. Second, edit-level attribution therefore makes moving a kernel to new hardware much cheaper than re-optimizing it, at equal quality.
What exists and what is missing
Several systems already run iterative refinement against real hardware, and they supply our machinery. CudaForge pairs a coder agent with a judge agent that reads 24 curated Nsight Compute counters. Hawkeye seeds the agent with ten expert unit tests per architecture and stacks optimizations across A100, H100, B200 and MI350. SwizzlePerf tunes workgroup-to-die mapping on the MI300X using L2 hit rate. GPU Kernel Scientist evolves MI300 GEMM kernels using only wall-clock time. EvoEngineer and KernelFoundry add population-based and quality-diversity search. Every one of them optimizes for a single device per run.
Two of these papers also give the strongest motivating evidence. EvoEngineer's authors note that kernels tuned on an RTX 4090 are not optimal on an H100. KernelFoundry ran crossover tests and found that SYCL code tuned for an Intel Arc B580 runs worse on an integrated Arc 140V. That shows bias survives even inside one vendor's open standard. KernelFoundry still re-optimizes per device and does not diagnose which parts of the code caused the gap.
HPCTransCompile is the closest work on moving kernels across hardware. It fine-tunes a model offline on compiler-generated CUDA-to-CPU pairs. It does not run an online loop and does not cover GPU-to-GPU porting.
TritonRL constrains our design. It shows that separating planning from implementation helps credit assignment, and that refinement loops reward-hack by wrapping PyTorch calls. Any ledger must therefore be checked with a cheating detector, or "transferable edits" will include fake ones.
CAKE separates schedule from layout at the language level. That is a cleaner fix in principle, but it is tied to NVIDIA and needs human merge gates. We work at the level of edits to ordinary source code instead.
Finally, the literature contradicts itself on how much feedback an agent needs. CudaForge and SwizzlePerf argue that agents need dense hardware counters. GPU Kernel Scientist uses zero profiling, and Hawkeye finds that an 11k-token prompt beats 1.2M tokens of documentation. Nobody has asked whether dense feedback makes kernels less portable. Our first study measures exactly that.
Research Program
The first component is a refinement loop with an edit ledger. It is CudaForge-style: a coder agent, a judge agent, and functional tests on randomized inputs. Every accepted change is stored as a minimal patch together with the agent's one-line rationale. We record four things for each patch: the order in which it was applied, the edits it depends on syntactically, its measured effect when added, and a TritonRL-style check that it does not call a vendor library.
The second component is per-edit attribution across devices. Let $t_d(K)$ be the median runtime of kernel $K$ on device $d$, over a set of input shapes that includes held-out shapes. For edit $e$ in final kernel $K$, the contribution on device $d$ is
$$ c_d(e) = \log t_d(K \setminus e) - \log t_d(K), $$
where $K \setminus e$ is the kernel with that patch reverted, plus any patches that depend on it.
An edit is portable if $c_d(e) > 0$ on both the source and target devices. It is source-specific if it is positive only on the source. It is harmful on the target if $c_{\text{target}}(e) < 0$.
Removing edits one at a time ignores interactions, so we check for them directly. We remove edits in reverse order of application. We then test the sum of single-edit contributions against the measured all-edit gap, and report how much of the gap the sum fails to explain. If this unexplained share exceeds roughly a third, we fall back to group-level attribution over the dependency clusters.
The third component is ledger-guided porting. Porting starts from the source kernel. It removes the edits that hurt on the target, keeps the portable ones, and gives the target-device refinement loop a short summary of what was removed and why. That summary tells the target loop to search where the source-specific edits used to be, rather than repeating the portable ones.
The fourth component is a study of feedback density and refinement depth. Feedback comes in three densities: wall-clock time only, the Hawkeye-style taxonomy prompt, and the full set of 24 profiler counters. We run the same loop at each density, take snapshots at iterations 1, 3, 10 and 30, and measure the share of total speedup that comes from portable edits. The hypothesis is that this share falls with depth and falls faster under dense feedback. That would fit Yan et al.'s observation that early iterations give the big, generic jumps.
Hardware pairs come in two tiers. The within-vendor pairs are RTX 4090 to H100, chosen because EvoEngineer reports this exact mismatch, and A100 to H100. The cross-vendor pair is H100 to MI300X. Kernels are moved across vendors with hipify, and kernels that depend on Hopper-only instructions are recorded as non-transferable rather than dropped. Where hardware is available we add the Intel B580 to 140V pair, reusing KernelFoundry's SYCL setting.
Baselines
The naive reference point has two parts. The first is the source-optimized kernel run unmodified on the target, hipified where needed. The second is PyTorch eager and torch.compile on the target. Together they show how much bias exists and whether porting is worth anything at all.
The strongest existing method is a full refinement run from scratch on the target device, configured as CudaForge. That paper reports 97.6% correctness and a 1.68× average speedup over PyTorch on KernelBench, rising to 2.27× at 30 iterations, all on NVIDIA. On the MI300X we instead use the GPU Kernel Scientist configuration, which reports about 450µs against PyTorch's ~850µs geometric-average GEMM time on the AMD Developer Challenge shapes. We give this baseline the same model, the same per-kernel token and wall-clock budget as the full pipeline (source search plus porting), and the same test harness. Ledger porting beats it if it reaches at least 95% of the from-scratch target speedup using no more than 30% of its target-side budget, or if it beats the from-scratch result at equal budget.
There is no reported result on Lacuna for cross-device kernel porting by a refinement agent. KernelFoundry's crossover result is only qualitative.
The warm-start baseline runs the same target loop starting from the source kernel, with no ledger. This is the direct ablation of our key ingredient, because it has everything except attribution.
Two further ablations isolate where any advantage comes from. One keeps a random subset of edits of the same size as the portable set, which tests whether attribution beats chance. The other removes harmful edits without running any new target search, which tests whether removal alone does the work.
The re-tuning baseline keeps the code structure fixed and re-sweeps only tile and block parameters on the target. It follows KernelFoundry's template-plus-sweep approach.
Why not just…?
Why not just re-run the search on each new target? This is what every current system does. It is the right answer when the target is available and compute is cheap. The proposal's value claim is about cost and about what we learn, not about a higher ceiling. If from-scratch search matches ledger porting at the same reduced budget, the method claim fails. Even then, the measurement study would still stand: it would show which kinds of optimization transfer. The equal-budget comparison and the budget-sweep curve settle this question.
Why not just warm-start refinement from the source kernel? This is the most serious threat, because a capable agent may notice and undo bad edits on its own. Our reason to doubt it comes from the known open problems of refinement loops. Agents settle on small variations of the strategy they start from, and KernelBench-style loops get no signal about which inherited choice causes a slowdown. The warm-start ablation decides this directly. If warm-start recovers more than 90% of the ledger's gain, we will report the ledger as a diagnostic tool rather than a porting method.
Why not just autotune the parameters, or write in a portable language? Re-tuning cannot undo structural choices such as a fusion order or a die-specific block mapping. KernelFoundry's B580 to 140V result shows that a portable language alone does not remove bias. Writing in Triton also costs correctness: KernelBench finds models write it worse than CUDA, and TritonRL finds widespread cheating. Re-tuning stays in the experiments as a baseline. The ledger also tells us what share of the gap is purely parametric, which directly measures how much of this objection holds.
Proposed Experiments
The kernel set has four parts. We take 60 KernelBench Level 1 and Level 2 tasks that compile under both CUDA and HIP, the 10 SwizzlePerf kernels, and the 18-shape AMD GEMM set. For every task we also hold out shapes beyond the tuning shapes.
The primary metric is retention. Retention is the target speedup of the ported kernel divided by the target speedup of the from-scratch search, both measured against target-side PyTorch eager. We report it alongside target budget spent and KernelBench's fast$_p$ correctness-plus-speed score. Secondary outputs are the portable share of speedup as a function of iteration and feedback density, the interaction residual, and portability to held-out shapes, which turns shape overfitting into a measured quantity.
The first decisive check is whether, under dense profiler feedback, the portable share falls clearly between iteration 3 and iteration 30. If it does not fall, hardware bias is not accumulating through refinement, and the motivation weakens. The second decisive check is whether the ledger porting beats warm-start at equal target budget on the cross-vendor pair. The third is whether the removal-only ablation already captures most of that gain. If it does, the cheap part of the method is the useful part.
References
- CudaForge: An Agent Framework with Hardware Feedback for CUDA Kernel Optimization
- Hawkeye: Hardware-Aware GPU Kernel Optimization with Minimal Supervision
- SwizzlePerf: Hardware-Aware LLMs for GPU Kernel Performance Optimization
- GPU Kernel Scientist: An LLM-Driven Framework for Iterative Kernel Optimization
- EvoEngineer: Mastering Automated CUDA Kernel Code Evolution with Large Language Models
- KernelFoundry: Hardware-Aware Evolutionary GPU Kernel Optimization
- HPCTransCompile: An AI Compiler Generated Dataset for High-Performance CUDA Transpilation and LLM Preliminary Exploration
- TritonRL: Training LLMs to Think and Code Triton Without Cheating
- CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution
- CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation
- KernelBench: Can LLMs Write Efficient GPU Kernels?