Correctness Firewalls for Synthesized Database Components

Mohammad Javad Amiri, Carsten Binnig, Andreas Kipf, Tim Kraska, Ryan Marcus, Johannes Wehrstein, Chenyuan Wu, Geoffrey X. Yu, Zijie ZhaoarXiv 2026 · arXiv

TL;DR

High-performance, "bespoke" database code is now cheap to generate thanks to AI agents, but it lacks the rigorous correctness guarantees of traditional systems. This paper proposes correctness firewalls—lightweight design patterns that allow untrusted, AI-generated components to optimize performance while ensuring the overall system never produces incorrect results.

What Problem Does This Paper Address?

Database management systems (DBMS) are the bedrock of modern infrastructure, requiring absolute correctness. Traditionally, this is achieved through decades of human engineering, rigorous testing, and "correctness-oriented" designs that often sacrifice raw performance for human interpretability.

Enter Large Language Model (LLM) coding agents. These agents can generate "instance-optimized" code—algorithms tailored specifically to a single dataset or workload—that significantly outperforms general-purpose systems. However, a purely agentic approach is risky. Agents can "reward hack" (fixing a symptom without the cause), introduce subtle logic bugs, or fail to handle edge cases. The authors tackle a fundamental question: How can we exploit the performance gains of AI-generated code without abandoning the hard-won correctness guarantees of the database community?

Methods

The authors propose a hybrid architecture where synthesized components are placed behind correctness firewalls. These firewalls ensure that even if the AI-generated code is buggy or suboptimal, the system's output remains valid. They define three primary firewall paradigms:

  1. Advisory (e.g., Cardinality Estimation): The agent provides a "hint" to the system. A bad hint might lead to a slower query plan, but it cannot change the final answer.
  2. Guarded (e.g., Learned Indexes): The agent's output is checked by a cheap, trusted "guard" function. If the output violates a known invariant, the system falls back to a slow-but-safe traditional algorithm (like binary search).
  3. Domain Certified (e.g., Data Compression): For components with small, finite input domains (like a single file), the synthesized code is exhaustively tested against every possible input before deployment. If it passes for that specific domain, it is "certified" for use.

Datasets

The researchers evaluated their approach using several standard and complex benchmarks:

  • Cardinality Estimation: The Join Order Benchmark (JOB) and the more difficult JOB-Complex.
  • Searching Sorted Data: Integer datasets (600M records, 4.8 GB) and string/URL datasets (105.9M records, 12 GB).
  • Data Compression: OpenStack log data and the Yelp dataset.

Key Insights and Ideas

  • Performance vs. Interpretability: Human-written code often uses simpler abstractions (like hash tables) to ensure robustness. Agents aren't bound by human readability, allowing them to find complex, "messy" optimizations that squeeze out every nanosecond of performance.
  • Semantic Awareness: Agents are surprisingly good at spotting semantic patterns. For example, in the compression case study, the agent noticed that log timestamps were sequential and used delta-encoding, a trick a general-purpose compressor might miss.
  • The "Cost of Correctness": The paper introduces the idea that there is a performance gap between a "free" agent (which might return wrong answers quickly) and a "constrained" agent behind a firewall. Quantifying this gap is a key future research direction.

Key Results

The early experiments show that AI-generated components behind firewalls can significantly outperform state-of-the-art human-engineered solutions:

  • Cardinality Estimation: Using GPT-5.4, the "Bespoke-Card" estimator beat PostgreSQL's default estimator by 33% on JOB and 68% on JOB-Complex, closing roughly 75% of the performance gap between standard estimates and "true" cardinalities.
Figure 1: Agent (GPT-5.4) generated cardinality estimators on JOB [60] and JOB-Complex [108] compared with Post- greSQL’s estimator and true cardinalities. Details in [110].
  • Searching Sorted Data: Synthesized indexes were up to 3x faster than human-designed learned indexes (like RMI or PGM) for integer data and showed significant gains for complex string data.
Figure 2: Size / latency tradeoff for indexing sorted data.
  • Data Compression: The agent-generated codecs achieved "Pareto dominance," meaning they were both faster and achieved better compression than industry standards like Zstd and LZ4. On the Yelp dataset, the agent found a codec with 50% better compression than Zstd at similar speeds.
Figure 3: Compression factor / decompression rate trade- off for compressing OpenStack logs [45] and Yelp [3] data, comparing agent (GPT-6-Astra) generated compression tech- niques, LZ4 [23], Zstandard [22], Brotli [5], and OnPair+ [93]. See [45] for details.

Limitations and Future Work

  • Safety Beyond Logic: Firewalls protect against logical errors, but they don't automatically prevent synthesized code from crashing the process (segfaults), looping forever, or maliciously modifying memory. The authors suggest using Domain-Specific Languages (DSLs) or sandboxing (like WebAssembly) to mitigate these risks.
  • Security: Trusting agents introduces new attack vectors like prompt injection or data poisoning. Future work must investigate how to sandbox these agents effectively.
  • Formal Methods: While the authors are skeptical of agent-written proofs today, they see a future where agents generate code that is then verified by "ironclad" human-written specifications using tools like Verus.
  • Human Insight: A major concern is that as agents take over code generation, humans might lose the intuition and knowledge needed to build the next generation of systems. The authors emphasize that we should use synthesized code as a way to "peek inside" and learn new techniques.

Summary updated

Abstract

For the first time in history, high-performance code is cheap. Today's coding agents can generate bespoke, instance-optimized data systems with excellent performance, and such agents will continue to improve as LLMs advance. But a purely agentic approach, even when validated with strong testing techniques, does not automatically inherit the correctness guarantees our community has spent decades building. We propose a middle ground between hand-crafted and agent-crafted systems: correctness firewalls, lightweight design patterns under which a synthesized component can affect a system's performance but never its results. Behind a correctness firewall, agents are free to synthesize fast components while the overall system remains correct. We present three case studies, each demonstrating a different type of firewall: cardinality estimation (advisory), searching sorted data (guarded), and data compression (certified). In early experiments, synthesized components search sorted data up to 3x faster than human-designed learned indexes and can compress text 1.5x smaller than Zstd. Drawing on these case studies, we outline a research agenda for how data management systems can exploit coding agents without giving up their hard-won correctness guarantees, from protecting synthesized code to measuring the cost of correctness.
BibTeX
@misc{amiri2026correctness,
  author = {Mohammad Javad Amiri and Carsten Binnig and Andreas Kipf and Tim Kraska and Ryan Marcus and Johannes Wehrstein and Chenyuan Wu and Geoffrey X. Yu and Zijie Zhao},
  title = {{Correctness Firewalls for Synthesized Database Components}},
  year = {2026},
  eprint = {2611.05173},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.05173},
}
Figure