FlashScript: Compiling Recurring Agent Workflows into Reusable Neuro-Symbolic Programs

Sadra Sabouri, Hosein Hasanbeig, Adam Sage, Shreya Sinha, Cale Darling, Ashish Tiwari, Katie Garms, Souti Chattopadhyay, Sumit GulwaniarXiv 2026 · arXiv

TL;DR

FlashScript introduces a method to compile inconsistent, expensive AI agent trajectories into reusable Neuro-Symbolic Programs (NSPs). By freezing successful actions into deterministic code and localizing LLM calls to specific "judgment" points, FlashScript achieves 100% consistency, 70x lower token costs, and significantly higher accuracy on complex, recurring workflows compared to traditional agents.

What Problem Does This Paper Address?

AI agents are increasingly used for recurring tasks, like generating weekly reports or monitoring data. However, current agents operate in a "reason-action" loop, regenerating their logic from scratch every time. This leads to three major issues:

  1. Inconsistency: The same prompt can yield different tool calls or output formats on different days.
  2. Cost: Re-running the full reasoning chain is computationally expensive and token-heavy.
  3. Brittleness: While "skills" (natural language instructions from past runs) help, agents still struggle with complex, multi-step tasks where errors can propagate.

The authors argue that a successful agent run contains valuable procedural knowledge that should be "compiled" into a stable software asset rather than just being used as a loose suggestion for the next run.

Methods

The core contribution is FlashScript, a set of guidelines for authoring NSPs from agent traces. The process transforms a "black-box" agent run into a structured script:

  • Actions to Code: Tool calls (like database queries or data merging) are converted into standard Python code.
  • Parameters: Instance-specific values (e.g., a specific city name) become function arguments.
  • Typed Neural Calls (llm<T>): Decisions requiring semantic judgment are isolated into calls that return a specific type (e.g., a Boolean for a scam check).
Compiling an agent run into a neuro-symbolic program

To optimize these programs, the authors identify three layers for neural calls—Input (parsing intent), Computation (semantic logic), and Output (natural language summary)—and apply three reduction techniques:

  1. Caching: Storing previously resolved values (e.g., mapping "LA" to "Los Angeles") behind a guard.
  2. Codifying: Replacing simple LLM judgments with heuristic rules or regex.
  3. Clubbing: Batching independent questions into a single LLM call to save on overhead.

Datasets

The researchers evaluated FlashScript using:

  • WorkBench: A benchmark of 120 analytics tasks (12 templates with 10 instances each) requiring data reading, computation, and plotting.
  • Composite Tasks: A newly created set of tasks combining up to 10 independent subtasks to test how agents handle increased complexity.
  • Generalization Set: 108 unseen instances of the same task templates to ensure the compiled programs weren't "overfit" to a single run.

Key Insights and Ideas

  • Localization, not Elimination: FlashScript doesn't try to remove the LLM's non-determinism entirely. Instead, it "boxes" it. By fixing the control flow in code, the non-deterministic neural calls can be tested, versioned, and reviewed independently of the program logic.
  • The "Skill" Ceiling: The paper reveals a significant weakness in "skill-guided" agents. While they perform well on simple tasks, their accuracy plummets on composite tasks (down to 23%) because they mix up instructions or omit subtasks. NSPs maintain 100% accuracy because their execution path is hard-coded.
  • NSPs as Software Assets: This approach shifts the view of AI agents from "chatbots that do things" to "program synthesizers." A successful interaction results in a piece of software that an end-user can own, maintain, and reuse.

Key Results

The quantitative benefits of NSPs over vanilla and skill-guided agents (using Claude Haiku 4.5) are stark:

  • Consistency: NSPs achieved 100% consistency in tool calls and outcomes across all runs. Vanilla agents only matched their first run 82% of the time.
  • Efficiency: NSPs used roughly 70 times fewer input tokens than agents. Because the NSP doesn't need to pass the entire conversation history for every step, the token usage remains nearly flat even as task complexity grows.
  • Speed: Runtime was reduced by approximately two-thirds, dropping from ~4.4 seconds to ~1.5 seconds for standard tasks.
Token usage scaling between agents and programs

Limitations and Future Work

The authors acknowledge that current benchmarks often have "clear-cut" outcomes. Real-world recurring work involves more nuanced decisions—interpreting vague terms like "suspicious" or weighing personal preferences.

Future work includes:

  • Nuanced Benchmarking: Creating datasets that specifically test the "judgment" boundary of neural calls.
  • User Interfaces: Developing ways for non-technical users to inspect and customize these compiled NSPs without needing to read Python code.
  • Dynamic Coverage: Building systems where the NSP can identify when a new request falls outside its "compiled" logic and automatically fall back to an agentic step to update itself.

Summary updated

Abstract

Many agent tasks recur with new data or small changes, yet agents regenerate their reasoning and actions on each run, producing different results. Distilling a prior run into a skill helps the agent avoid earlier mistakes, but it still re-runs the full process each time, so runs can vary and incur that cost. We compile a successful agent run into a reusable neuro-symbolic program (NSP) with a set of guidelines we call FlashScript. Actions and data transformations become code, instance-specific values become parameters, and decisions that need model judgment become LLM calls. We identify three layers where these calls belong and three techniques that replace most of them with code. The compilation does not remove nondeterminism; it localizes it, so the deterministic program and its neural calls can be reviewed and tested separately. Across 120 WorkBench analytics tasks with Claude Haiku 4.5, the NSPs were correct and made identical tool calls on every run, using about 70 times fewer input tokens and a third of the runtime of a skill-guided agent whose runs were inconsistent. In a composite version of the tasks, the skill-guided agent's accuracy fell to 23%, and its output inconsistency increased, while the NSPs remained correct and identical across all runs. The NSPs successfully generalized to all 108 unseen instances of the same templates. Our findings suggest that NSPs from agent trajectories can be considered cumulative software assets for end-users to test, maintain, and reuse in their recurring workflows.
BibTeX
@misc{sabouri2026flashscript,
  author = {Sadra Sabouri and Hosein Hasanbeig and Adam Sage and Shreya Sinha and Cale Darling and Ashish Tiwari and Katie Garms and Souti Chattopadhyay and Sumit Gulwani},
  title = {{FlashScript: Compiling Recurring Agent Workflows into Reusable Neuro-Symbolic Programs}},
  year = {2026},
  eprint = {2610.33876},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.33876},
}
Figure