LLM4Fluid: Large Language Models as Generalizable Neural Solvers for Fluid Dynamics

Qisong Xiao, Xinhai Chen, Qinglin Wang, Xiaowei Guo, Binglin Wang, Weifeng Chen, Zhichao Wang, Yunfei Liu, Rui Xia, Hang Zou, Gencheng Liu, Shuai Li, Jie Liu

TL;DR

LLM4Fluid transforms Large Language Models (LLMs) into generalizable neural solvers for fluid dynamics by treating flow sequences as a "language" of physics. By combining physics-informed latent compression with a novel modality alignment strategy, the framework achieves state-of-the-art accuracy and demonstrates remarkable zero-shot and in-context learning capabilities across diverse flow scenarios without requiring retraining.

What Problem Does This Paper Address?

Simulating fluid dynamics is a cornerstone of engineering, from designing fuel-efficient aircraft to predicting ocean currents. Traditionally, this relies on Computational Fluid Dynamics (CFD), which is accurate but computationally expensive and slow. While deep learning models have emerged as a faster alternative, they suffer from a "generalization gap": a model trained on one specific flow (e.g., low-speed air) often fails completely when applied to another (e.g., high-speed turbulence) unless it is painstakingly retrained.

The authors identify two main hurdles:

  1. Feature Entanglement: Standard neural networks often produce "messy" internal representations where different physical features are jumbled together, making it hard for the model to learn universal physical laws.
  2. The Modality Gap: While LLMs are great at sequences, there is a fundamental mismatch between textual prompts (semantic data) and numerical flow fields (physical data), which leads to unstable predictions when simply "plugging in" an LLM.

Methods

LLM4Fluid uses a two-stage pipeline to bridge the gap between natural language processing and fluid mechanics:

  1. Disentangled Reduced-Order Modeling (ROM): To handle high-dimensional flow data, the authors first compress it into a compact latent space. They introduce a physics-informed disentanglement mechanism (using a specific loss function) that forces the encoder to represent flow features as near-orthogonal, independent modes. This ensures the "vocabulary" fed to the LLM is physically meaningful and structured.
  2. LLM as a Temporal Processor: The framework uses a pre-trained, frozen LLM (like OPT-6.7B or GPT-2) to predict how these latent features evolve over time.
  3. Modality Alignment: Instead of just concatenating text prompts to the data (prefix prompting), they embed textual context (e.g., "This is a Kolmogorov flow at Re=100") as positional embeddings. This aligns the semantic meaning of the scenario with the physical sequence, preventing the "modality mismatch" that usually degrades long-term accuracy.

Datasets

The authors evaluated LLM4Fluid on five diverse 2D flow datasets:

  • Kolmogorov Flow (Low-Re & High-Re): Standard benchmarks for studying transition to turbulence.
  • Lid-driven Cavity Flow: Flow in a container with a moving lid, testing boundary condition handling.
  • Channel Flow: Turbulent flow between two plates.
  • Dam-break Flow: A transient, high-intensity scenario involving a collapsing column of fluid.

These datasets provide a rigorous test of the model's ability to handle different Reynolds numbers, geometries, and physical behaviors.

Key Insights and Ideas

  • Physics as Language: The core "aha moment" is that the pre-trained sequence priors in LLMs—honed on billions of lines of text—are surprisingly effective at capturing the underlying "grammar" of physical evolution, provided the input is properly compressed and aligned.
  • The Power of Disentanglement: The authors found that without the physics-informed disentanglement loss, the latent space becomes chaotic, and the LLM fails to generalize. Forcing the model to separate physical modes is what allows it to act as a "universal" solver.
  • In-Context Learning for Physics: Much like how ChatGPT can learn a new task from a few examples, LLM4Fluid exhibits In-Context Learning (ICL). By providing a few "demonstration patches" of a new flow scenario at inference time, the model's accuracy improves significantly without any weight updates.

Key Results

  • Superior Accuracy: On the complex Dam-break dataset, LLM4Fluid reduced Mean Absolute Error (MAE) by 33.33% and Mean Squared Error (MSE) by 59.67% compared to the previous state-of-the-art (Time-LLM).
  • Zero-Shot Generalization: The model can be trained on one scenario (e.g., Low-Re Kolmogorov flow) and successfully predict a completely different one (e.g., High-Re) with much higher fidelity than competing models.
  • Efficiency Gains: Despite using large backbones, the framework is highly efficient. It is 9.9x faster in training and 8.2x faster in inference than Time-LLM, largely because it uses a more efficient tokenization and alignment strategy.
  • Scaling Laws: Accuracy consistently improves as the size of the LLM backbone increases (from 125M to 6.7B parameters), suggesting that larger "foundation models" for physics are a viable path forward.

Limitations and Future Work

While impressive, the authors note a few key limitations:

  • 2D Constraints: The current work is limited to 2D flow fields. Real-world engineering often requires 3D modeling, which introduces significantly higher dimensionality.
  • Fixed Resolution: The framework currently operates on a fixed spatial resolution (128x128).
  • Future Directions: The team plans to extend the model to 3D systems and multi-physics scenarios (e.g., coupling fluid flow with heat transfer) and explore ways to integrate governing equations (like Navier-Stokes) directly into the latent dynamics to further improve physical consistency.

Summary updated

Abstract

Deep learning has emerged as a promising paradigm for spatio-temporal modeling of fluid dynamics. However, existing approaches often suffer from limited generalization to unseen flow conditions and typically require retraining when applied to new scenarios. In this paper, we present LLM4Fluid, a spatio-temporal prediction framework that leverages Large Language Models (LLMs) as generalizable neural solvers for fluid dynamics. The framework first compresses high-dimensional flow fields into a compact latent space via reduced-order modeling enhanced with a physics-informed disentanglement mechanism, effectively mitigating spatial feature entanglement while preserving essential flow structures. A pretrained LLM then serves as a temporal processor, autoregressively predicting the dynamics of physical sequences with time series prompts. To bridge the modality gap between prompts and physical sequences, which can otherwise degrade prediction accuracy, we propose a dedicated modality alignment strategy that resolves representational mismatch and stabilizes long-term prediction. Extensive experiments across diverse flow scenarios demonstrate that LLM4Fluid functions as a robust and generalizable neural solver without retraining, achieving state-of-the-art accuracy while exhibiting powerful zero-shot and in-context learning capabilities. Code and datasets are publicly available at https://github.com/qisongxiao/LLM4Fluid.
BibTeX
@misc{xiao2026llm,
  author = {Qisong Xiao and Xinhai Chen and Qinglin Wang and Xiaowei Guo and Binglin Wang and Weifeng Chen and Zhichao Wang and Yunfei Liu and Rui Xia and Hang Zou and Gencheng Liu and Shuai Li and Jie Liu},
  title = {{LLM4Fluid: Large Language Models as Generalizable Neural Solvers for Fluid Dynamics}},
  year = {2026},
  eprint = {2601.21681},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.21681},
}
Figure