Toward Compiler World Models: Learning Latent Dynamics for Efficient Tensor Program Search

Haolin Pan, Lianghong Huang, Xvlin Zhou, Mingjie Xing, Yanjun WuarXiv 2026 · arXiv

TL;DR

This paper introduces a "world-model-inspired" framework for tensor program optimization that treats code scheduling as a sequence of action-conditioned latent state transitions rather than a series of static code snapshots. By rolling out scheduling decisions entirely within a continuous latent space, the framework captures complex transformation dependencies and filters out syntactic noise, achieving search quality comparable to state-of-the-art auto-schedulers like Ansor while requiring 10× fewer expensive hardware measurements.

What Problem Does This Paper Address?

To run deep learning models efficiently across diverse hardware (CPUs, GPUs, accelerators), tensor compilers must find the optimal "schedule"—the exact sequence of loop transformations, tiling, memory placements, and parallelization choices for a given workload. Because this search space is combinatorially massive and direct on-device measurement is slow, compilers rely on learned cost models to evaluate and rank candidate programs.

However, existing cost models suffer from a fundamental flaw: they treat candidate programs as static code snapshots, completely ignoring the step-by-step transformation history that generated them.

This static view causes two major issues:

  1. Syntax Sensitivity: Minor variations in workload sizes (e.g., a matrix dimension of 1023 vs. 1024) can force the compiler to generate extra boundary-handling loops ("loop peeling"). Even though the core computation and hardware behaviors are virtually identical, the resulting code texts look completely different, misleading static code encoders.
  2. Referential Ambiguity: Scheduling actions are context-dependent. For example, an action might fuse two loops to create a new loop variable $k$, which a subsequent action then tiles. A static or action-only model cannot easily resolve what entity $k$ actually represents without tracking the intermediate states.
Different approaches to schedule evaluation.

Methods

The authors address this by framing tensor program optimization as a world model problem, where the compiler and hardware act as the environment, TensorIR representations act as states, and scheduling choices act as actions.

Instead of mutating the actual Abstract Syntax Tree (AST) or generating text at every step, the framework simulates the entire scheduling trajectory in a continuous latent space using three core components:

  1. TensorIR State Encoder: A trainable CodeBERT-based model adapted for long inputs via a sliding-window chunking mechanism. It uses contrastive learning to map initial unoptimized program states ($s_0$) into a dense latent space ($z_0$).
  2. Action-Conditioned Multi-Step Transition Model: Inspired by knowledge graph representation learning (specifically TransH), this lightweight geometric model recurrently applies scheduling actions directly to the latent state ($z_{t+1} = F(z_t, a_t)$). This allows the model to efficiently predict the final terminal-state representation ($\hat{z}_T$) while tracking the provenance of every decision.
  3. Ranking-Based Cost Model: An XGBoost model trained with a LambdaRank objective. Instead of predicting absolute execution time, it takes the predicted terminal latent state, structured action features, and hardware features to rank candidate schedules within the same workload.
Overview of the compiler world model framework.

Datasets

The framework was trained and evaluated using:

  • TenSet Tuning Logs: A large-scale program performance dataset for tensor compilers. The authors extracted 10,000 samples for contrastive state-representation learning, nearly 400,000 state-action pairs for transition modeling, and up to 500,000 measured samples for the ranking model.
  • Evaluation Benchmarks: 103 subgraphs extracted from 7 representative deep learning models covering CNNs, Transformers, and 3D vision (including ResNet-50, MobileNetV2, BERT, GPT-2, and OPT-1.3B). These were further grouped into 22 representative subgraph types for fine-grained analysis.

Key Insights and Ideas

  • Latent Space Rollouts Bypass Compiler Overhead: Simulating code transformations mathematically in a latent space avoids the heavy computational overhead of materializing intermediate ASTs or text strings during active search.
  • Suppression of Syntactic Noise: By learning transition dynamics rather than static text patterns, the model learns to group semantically equivalent programs together, making it robust against superficial syntax changes like loop peeling.
  • Prioritizing Ranking over Regression: Predicting absolute execution times across different hardware and workloads is notoriously difficult. Shifting the objective to intra-workload ranking perfectly aligns the model with the auto-scheduler's actual task: prioritizing the best candidates for physical measurement.

Key Results

The framework was integrated into TVM AutoScheduler and tested on an Intel Xeon CPU and an NVIDIA RTX 4090 GPU:

  • High Sample Efficiency: With a budget of just 1,000 trials, the method matched the search quality of Ansor running for 10,000 trials (within 2.2% geometric mean latency), yielding a 10× reduction in tuning measurements.
  • Substantial Speedups: Under a strict 64-trial budget, it improved representative-subgraph latency over Ansor by 1.37× on GPU and 1.54× on CPU.
  • End-to-End Acceleration: For full-model inference, the optimized programs outperformed standard PyTorch by 4.61× and cuDNN-enabled PyTorch-opt by 3.67× (geometric mean).

Limitations and Future Work

  • Bounded by Candidate Generation: This framework is purely an evaluator. If the underlying auto-scheduler fails to propose high-quality candidate schedules in its search space, a better evaluator cannot magically recover them.
  • Compounding Errors: Because the terminal state is reached via multi-step rollouts, prediction errors can accumulate over exceptionally long or complex scheduling trajectories.
  • No Cross-Workload Calibration: Because the model uses a relative ranking objective within individual workloads, its scores cannot be used to compare absolute performance across entirely different models or distinct hardware targets.

Summary updated

Abstract

Tensor program optimization is essential for modern machine learning systems, but its search space is enormous. Existing auto-schedulers reduce measurement cost with learned cost models, yet they usually evaluate each candidate as a static code snapshot, ignoring the schedule trajectory that produced it. This makes them insensitive to action dependencies and vulnerable to superficial code variations. We propose a \emph{world-model-inspired} evaluator that models schedule evaluation as action-conditioned latent dynamics over program states. Starting from the initial program, it rolls out scheduling actions in a continuous latent space with a lightweight transition model, avoiding expensive AST mutation and repeated code encoding. The final dynamic representation is combined with action and hardware features to rank candidates. Implemented in TVM AutoScheduler, our method improves representative-subgraph latency over Ansor by 1.37$\times$ on GPU and 1.54$\times$ on CPU under the same 64-trial budget. It also matches Ansor-10K within 2.2% geometric mean using 10$\times$ fewer measurements, and accelerates full-model inference over PyTorch/PyTorch-opt(cuDNN) by 4.61$\times$/3.67$\times$ geometric mean.
BibTeX
@misc{pan2026toward,
  author = {Haolin Pan and Lianghong Huang and Xvlin Zhou and Mingjie Xing and Yanjun Wu},
  title = {{Toward Compiler World Models: Learning Latent Dynamics for Efficient Tensor Program Search}},
  year = {2026},
  eprint = {2606.09312},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.09312},
}

Loading related content...

Figure