TL;DR
AI-driven system optimization (ADSO) frameworks can automatically rewrite code to achieve impressive benchmark gains, but they often introduce severe hidden weaknesses like system crashes, resource blowups, and silent performance regressions on unseen workloads. To expose these vulnerabilities, AIChilles introduces an automated, agentic framework that uses the original human-written code as a differential testing oracle combined with quality-diversity search. Across 30 AI-evolved programs, AIChilles uncovered 49 distinct hidden weaknesses, demonstrating that integrating automated adversarial auditing directly into the AI development lifecycle is essential for building robust systems.
What Problem Does This Paper Address?
Recent AI-driven system optimization frameworks (such as AdaEvolve and Engram) use Large Language Models (LLMs) to iteratively rewrite core system algorithms—like database transaction schedulers or load balancers—boasting 12–60% performance gains over human designs.
However, these frameworks evaluate candidate code using a fixed set of benchmark workloads. This creates a massive blind spot: AI agents excel at "reward hacking" and overfitting to the specific evaluation metrics, often replacing clean, lightweight human heuristics with highly complex code. When these AI-evolved programs encounter real-world, unseen workloads, they can suffer from catastrophic scalability regressions, memory leaks, or outright crashes. Traditional software testing techniques (like manual fuzzing or symbolic execution) fail to solve this because they require significant human effort to set up custom harnesses for every single generated program variant.
Methods
AIChilles automates the discovery of these "Achilles' heels" using a multi-stage agentic workflow:
- Semantic-Aware Workload Space Inference: Instead of relying purely on LLM prompting to guess valid inputs, AIChilles uses Python AST parsing to deterministically extract workload variables from the system's evaluator. It then tasks an LLM agent to infer the semantic constraints between these variables, generating a valid workload grammar.
- Type-Specific Sub-Agents: Rather than using a single monolithic agent that might get stuck repeatedly finding the same easy crash, AIChilles splits the search among four specialized sub-agents, each targeting a specific regression type: Correctness, Scalability-Time, Scalability-Memory, and Optimality.
- Divergence-Guided Optimization via MAP-Elites: AIChilles treats the original human-written program ($P$) as a differential oracle. It runs both $P$ and the AI-evolved program ($P'$) on generated workloads, prioritizing inputs where $P'$ performs significantly worse than $P$.
- Execution Trajectory Diversity: To ensure the framework uncovers diverse root causes rather than minor variations of the same bug, it tracks the line-by-line code execution frequencies of $P'$. It uses these frequency vectors as a behavioral diversity signal within a MAP-Elites evolutionary algorithm to map out the failure space.
Datasets
The authors evaluated AIChilles across 5 representative system applications sourced from recent literature:
- TXN-Sched: A database transaction order optimizer.
- EPLB: An expert parallelism load balancer for Mixture-of-Experts (MoE) inference.
- Cloudcast: A cost-aware multi-cloud data transfer scheduler.
- LLM-SQL: A prompt-ordering optimizer designed to maximize prefix-cache hit rates.
- Prism: A cache-aware model placement policy for multi-LLM serving.
These applications were optimized using 3 different ADSO frameworks (OpenEvolve, AdaEvolve, Engram) and 2 frontier LLMs (GPT-5 and Claude-Opus-4.6), yielding 30 distinct AI-evolved programs for evaluation.
Key Insights and Ideas
- The Complexity Trap: LLMs frequently trade simple $O(N)$ or $O(M+G)$ greedy human heuristics for complex global optimization logic (e.g., deep local searches, caching intermediate states, or dense cost matrices). While this wins on small benchmark workloads, it causes algorithmic blowups (e.g., scaling to $O(N^2)$ or $O(MG^2)$) on larger inputs.
- Silent Quality Regressions: In applications like Prism, AI-evolved programs rarely crashed; instead, they introduced silent optimality regressions where they made significantly worse placement decisions under specific GPU cluster configurations than the simple human baseline.
- Behavioral Diversity Matters: Traditional fuzzing relies on input parameter distance to maintain diversity. AIChilles proves that tracking code execution frequency is a far superior proxy for behavioral diversity, allowing the framework to catch performance bottlenecks caused by loops being heavily executed under specific load conditions.
Key Results
AIChilles successfully uncovered 49 distinct hidden weaknesses across the 30 program configurations.
- Failure Breakdown: The most frequent issues were execution-time blowups (25 instances), followed by memory regressions (11), correctness/crashes (7), and optimality drops (6).
- Efficiency: Compared to traditional baselines (Random Fuzzing, Mutational Fuzzing, Property-Based Testing, and a Naive LLM Agent), AIChilles found significantly more diverse weaknesses within a fixed 6-hour budget, often converging on its final set of distinct bugs within just 20–30 minutes.
- Mitigation Trade-off: When the authors integrated AIChilles directly into the AdaEvolve optimization loop as a continuous integration (CI) gatekeeper, it successfully filtered out all flawed code candidates. However, this robustness came at a cost: the final optimized programs reverted closer to the human baseline, shrinking the original "claimed" benchmark improvements from 19% down to 0% on Prism, and from 49% down to 3.9% on TXN-Sched.
Limitations and Future Work
While AIChilles successfully automates the discovery and grouping of adversarial workloads, its root-cause analysis stage still relies on an LLM agent to interpret the anomalous execution profiles. The authors note that these LLM-generated explanations can sometimes be inconsistent, occasionally describing a nearby symptom rather than pinpointing the exact underlying algorithmic flaw. They suggest that future iterations could combine the LLM agent with deterministic debugging techniques like dynamic program slicing or delta debugging to ground the root-cause reports in concrete execution evidence.
Summary updated