Abstract
MLIR (Multi-Level Intermediate Representation) has rapidly become a foundational technology for modern compiler frameworks, enabling extensibility across diverse domains. However, ensuring the correctness and robustness of MLIR itself remains challenging. Existing fuzzing approaches—based on manually crafted templates or rule-based mutations—struggle to generate sufficiently diverse and semantically valid test cases, making it difficult to expose subtle or deep-seated bugs within MLIR's complex and evolving code space. In this paper, we present FLEX, a novel self-adaptive fuzzing framework for MLIR. FLEX leverages neural networks for program generation, a perturbed sampling strategy to encourage diversity, and a feedback-driven augmentation loop that iteratively improves its model using both crashing and non-crashing test cases. Starting from a limited seed corpus, FLEX progressively learns valid syntax and semantics and autonomously produces high-quality test inputs. We evaluate FLEX on the upstream MLIR compiler against four state-of-the-art fuzzers. In a 30-day campaign, FLEX discovers 80 previously unknown bugs—including multiple new root causes and parser bugs—while in 24-hour fixed-revision comparisons, it detects 53 bugs (over 3.5x as many as the best baseline) and achieves 28.2% code coverage, outperforming the next-best tool by 42%. Ablation studies further confirm the critical role of both perturbed generation and diversity augmentation in FLEX's effectiveness.
Introduction
Compilers serve as the bedrock of software development. MLIR has emerged as an industry standard for building these compilers, particularly in domains like AI and hardware design. Its "dialect" system allows for a vast range of representations, from high-level mathematical operations to low-level machine code. However, this flexibility introduces significant complexity, making MLIR testing a formidable challenge.
Traditional fuzzers often rely on rigid, hard-coded templates or mutation rules that fail to keep pace with MLIR's rapid evolution. Furthermore, while Large Language Models (LLMs) show promise in code generation, the scarcity of high-quality, diverse MLIR training data limits their effectiveness. FLEX addresses these issues by interleaving LLM training with active exploration, creating a fuzzer that improves its generative capabilities through continuous feedback from the compiler.
The FLEX Framework
FLEX implements a self-adaptive loop designed to bootstrap its way out of the data scarcity trap. The architecture consists of four primary stages:
- Training (LoRA): FLEX utilizes a pre-trained CodeGen-2B model as its base. To adapt it to MLIR efficiently, it employs Low-Rank Adaptation (LoRA), which fine-tunes the model on a small set of initial seeds without the prohibitive cost of full retraining.
- Perturbed Generation: To prevent the model from becoming overly deterministic, FLEX introduces probabilistic perturbations during decoding. By adjusting the sampling temperature, the system is forced to explore unusual and edge-case code paths.
- Compiling and Feedback: Generated programs are processed by the MLIR compiler. Crashes are reported as bugs, while valid programs provide semantic feedback.
- Diversity Augmentation: Syntactically valid programs that do not cause crashes are fed back into the training set. This "virtuous cycle" allows the model to autonomously expand its vocabulary and learn complex semantics not present in the original seed data.
Evaluation and Results
FLEX was evaluated against four state-of-the-art MLIR fuzzers: MLIRSmith, MLIRod, SynthFuzz, and Ratte. The initial seed corpus consisted of 15,344 programs from the official LLVM project.
Bug Discovery and Coverage
In a 30-day campaign, FLEX identified 80 previously unknown bugs, 44 of which have already been fixed by developers. In a controlled 24-hour comparison, FLEX significantly outperformed all baselines, detecting 53 unique bugs—over 3.5 times more than the nearest competitor, MLIRod.
FLEX also achieved a line coverage of 28.2%, a 42% improvement over the best-performing baseline. This increased coverage is attributed to the model's ability to generate diverse and semantically valid code that reaches deep into the compiler's logic. Analysis of the bug distribution shows that 87% of the bugs FLEX detected were exclusive to it, highlighting its ability to explore logic unreachable by rule-based systems.
Efficiency and Triage
FLEX maintains high efficiency, executing approximately 13.8 tests per minute. Unlike template-based fuzzers that often produce bloated test cases (up to 40,000 tokens), FLEX generates compact programs averaging 146 tokens. This compactness significantly reduces the time required for bug triage, allowing developers to isolate root causes more quickly.
Conclusion
FLEX demonstrates the power of interleaved learning and exploration in compiler testing. By treating its own successful outputs as training data, FLEX overcomes the limitations of static rule-based systems and the data scarcity of specialized IRs. While it requires GPU resources for its iterative loop, the resulting gains in bug discovery and code coverage represent a significant advancement in automated software testing. Future work will explore the application of this self-adaptive framework to other domain-specific languages where training data is scarce.
Summary updated