Setting up a multi-physics simulation—like modeling how CO2 moves through underground rock layers—usually requires a PhD and hours of tedious manual configuration. You have to juggle complex XML input "decks," ensure material properties are consistent across different physical solvers, and troubleshoot cryptic error logs when the simulation inevitably crashes.
This paper from researchers at Stanford University introduces an agentic harness designed to automate this process. By splitting the work between Large Language Models (LLMs) and deterministic code, they’ve created a system that can take a natural language description and produce a validated, physically consistent simulation.
TL;DR
The authors present a framework that uses LLMs for reasoning and deterministic code for "ground truth" construction to automate multi-physics simulations. By separating task allocation, they achieved 100% reliability in generating valid simulation inputs for complex carbon storage problems, moving from natural language prompts to verified results without human intervention.
What Problem Does This Paper Address?
LLMs are surprisingly good at writing code and configuration files, but they are notoriously unreliable for high-stakes engineering. In the context of multi-physics simulators like GEOS, an LLM might generate an input file that looks correct but uses obsolete syntax, violates the laws of thermodynamics, or simply ignores half of the user's instructions.
The challenge is that a simulation can fail in four main ways:
- Syntactic: The simulator can't read the file.
- Physical Inconsistency: The inputs (like pressure and density) don't make physical sense together.
- Fragile Repairs: Fixing one error might break a part of the file that was already working.
- Silent Failures: The simulation finishes, but it modeled the wrong thing (e.g., the wrong injection rate).
Methods
The researchers built an "agentic harness" using LangGraph to orchestrate the workflow. The core philosophy is explicit task allocation:
- The LLM (Claude 3.5 Sonnet / Llama 3): Handles "fuzzy" tasks like interpreting the user's request, browsing documentation, and diagnosing error logs to propose local repairs.
- Deterministic Routines: Handle "precise" tasks. These are hard-coded rules that calculate hydrostatic pressure, convert units, and verify the XML schema.
The workflow follows an interpret–act–validate loop, as shown in the coordination diagram:
The system also uses grounding, providing the LLM with up-to-date documentation and reference "decks" to prevent it from hallucinating outdated parameters. Finally, it employs a physics gate—a suite of automated checks that verify if the simulation results conserve mass and stay within realistic physical bounds.
The specific breakdown of responsibilities is detailed in the table below:
Datasets
The framework was evaluated using four distinct CO2 storage scenarios (T1 through T4) using the GEOS simulator:
- T1 & T2: Basic flow problems (plume migration and well-pair sweeps).
- T3: Poromechanics involving continuum fault zones.
- T4: A complex scenario involving frictional contact on three faults, inspired by the SPE11C benchmark.
The authors also tested the portability of their "deterministic synthesis" approach using OpenFOAM (a CFD simulator) for dam-break and heat transfer cases.
Key Insights and Ideas
- Grounding is necessary but not sufficient: Providing the LLM with documentation (grounding) improved the "look" of the files, raising structural validity from 25% to 100%. However, the simulations only actually passed physics checks when the deterministic code took over the calculation of initial states and well controls.
- Controlled Local Repair: Instead of asking the LLM to "fix the file," the harness identifies the specific line causing an error and asks the LLM for a local edit. This prevents the agent from accidentally deleting valid parts of the setup to bypass an error.
- Fidelity vs. Physics: A major "aha moment" in the paper is the distinction between a simulation being physically consistent and faithful to the request. A simulation can conserve mass perfectly (passing the physics gate) while using the wrong injection rate (failing the fidelity check). You need both.
Key Results
The results highlight a massive jump in reliability when moving from "LLM-only" to the "Agentic Harness":
- 100% Acceptance: In 80 attempts across two flow problems, the full system (grounding + normalization + synthesis) achieved a 20/20 success rate for each, whereas ungrounded LLM generation failed every single time.
- Ablation Success: As shown in Figure 3, grounding fixes the "shape" of the data, but "Synthesis" (the deterministic physics rules) is what actually makes the simulations run.
- Complex Physics: The system successfully set up and ran a 45-day simulation of CO2 injection with frictional fault contact (T4) in just 10.9 minutes without any human intervention.
Limitations and Future Work
- Traceability: While the system works, it’s currently hard to "audit" exactly why a specific value was chosen (was it the LLM's guess or a deterministic rule?). The authors suggest a "change record" for better transparency.
- Generalization: The deterministic rules are currently specific to the GEOS and OpenFOAM simulators. Expanding this to a "universal" simulation agent would require a more generalized way to encode physics rules.
- Numerical Calibration: The current system checks if a simulation is consistent, but it doesn't yet automatically optimize for numerical accuracy (like refining the mesh or adjusting time steps for better convergence).
Summary updated