Satellites measure how hot the ground's skin is. People, trees and crops live in the air two meters above it. The two temperatures can differ sharply: after tree restoration, air temperature changes by only 15% to 30% as much as surface temperature does (Observed different impacts of potential tree restoration on local surface and air temperature). The gap comes from a few hidden physical quantities: how easily heat moves from surface to air (aerodynamic resistance), how much of the incoming energy goes into evaporation rather than heating, and how much the surface stores. Recovering those quantities from observations is an inverse physics problem. A recent line of work hands such problems to large language models (LLMs): the model proposes parameter values, a simulator checks them, and the model revises. The approach works because, the authors argue, LLMs carry "implicit world knowledge" of physics. A separate literature shows that LLM reasoning often collapses once familiar words are replaced with meaningless symbols. Nobody has asked whether the physics knowledge that helps LLMs in inverse problems is the same kind of word-triggered recall.
This proposal builds a controlled testbed for inferring surface-to-air heat exchange parameters, in which the variable names given to the model can be canonical, scrambled, or deliberately misleading, and the true parameters can sit inside or outside the range that the names suggest. The central test is whether an LLM-in-the-loop estimator does any better than a standard black-box optimizer that starts from a prior extracted from the LLM once, before search begins. If it does not, the LLM's contribution to inverse physics is a lexical prior and not a reasoning process. That has practical consequences: it tells us to query the model once and then optimize, and it tells us to distrust the model whenever a site's labels mislead, for example a "forest" pixel that is actually degraded.
Core Claim
We hypothesize that the advantage LLM-guided search holds over gradient-free optimizers in physical parameter identification comes almost entirely from priors keyed to variable names. It should therefore disappear under symbol obfuscation, reverse under misleading labels, and be fully reproduced by a one-shot LLM-elicited prior fed to Bayesian optimization. The thermodynamic setting matters for two reasons. It lies outside the rigid-body tabletop domains where these claims have been tested so far. It also has a structural trap: aerodynamic resistance and the evaporative share of energy can partly trade off against each other, so a system that reasons from feedback, rather than recalling typical values, should be able to detect when the data cannot tell them apart. Either outcome is informative. If the LLM loop keeps an advantage under obfuscation, that is the first evidence that LLMs contribute something to inverse physics beyond vocabulary.
Prior Work and Where This Sits
Three bodies of work motivate the proposal. The first is inverse physics with LLMs. LLMPhy has an LLM write simulator code with friction, mass and stiffness values, runs it in PyBullet, and feeds trajectory errors back as text. It beats Bayesian optimization and CMA-ES by 6% to 12% in reasoning accuracy on tabletop rigid-body data. DiffWind uses a multimodal LLM to guess material stiffness inside a differentiable wind-and-deformation model. Both treat the LLM's physics prior as trustworthy, and neither varies the names it sees. We reuse LLMPhy's generate-simulate-critique loop as our main LLM method and add exactly the manipulation it never ran.
The second is semantic dependency. On PlanBench, GPT-4 plans correctly on 34.3% of Blocksworld problems and on 2% when predicates are renamed with random strings (Large Language Models Still Can't Plan). On Zebra puzzles, GPT-4o falls from about 90% to about 25% under lexical obfuscation (Lexical Recall or Logical Reasoning). Reasoning models such as o1 still drop sharply under renaming (LLMs Still Can't Plan; Can LRMs?). These results supply the experimental manipulation, and the PrOntoQA "anti-sense" split used in Causality-Aware Post-Training supplies the idea of a misleading-label condition. The LLM-Modulo position paper sets a constraint. It shows that self-critique without an external verifier degrades answers, so every LLM condition here receives feedback from a sound simulator. It also warns that the loop starves when the generator's first proposals are hopeless, which is exactly what we expect under obfuscation.
The third is surface-to-air temperature modeling, which supplies the application and real data. UrbanHeat predicts street-scale air temperature from multi-resolution imagery by combining a coarse-climate scaling loss with a surface-temperature ranking loss. DiffTemp treats the surface-to-air transition as a conditional diffusion process on LSTAT-20K. Neither estimates physical parameters. The ExPhy benchmark constrains the design: it showed that good trajectory prediction does not imply correct parameters, so we score parameter recovery directly and not only the reconstruction of air temperature.
Research Program
The forward model is a deliberately simple single-column surface energy balance, so that ground truth is exact and identifiability can be analyzed. Net radiation $R_n$ splits into sensible heat $H$, evaporative heat $\lambda E$ and storage $G$. Air temperature follows from the bulk transfer relation
$$T_a = T_s - \frac{r_a H}{\rho c_p}, \qquad H = (1-\epsilon)(R_n - G), \quad G = \gamma R_n,$$
where $T_s$ is surface temperature, $r_a$ is aerodynamic resistance (s/m), $\epsilon$ is the evaporative fraction, $\gamma$ is the storage fraction, and $\rho c_p$ is the volumetric heat capacity of air. The hidden parameters are $\theta = (r_a, \epsilon, \gamma)$, plus albedo when $R_n$ is built from shortwave forcing. Each task gives an estimator a week of hourly forcing and observed $T_s$ and $T_a$ series with added noise, plus a verbal site description, and asks it to recover $\theta$ within a fixed budget of simulator calls. Because $T_s - T_a$ depends on the product $r_a(1-\epsilon)$ when $\gamma$ is small, we generate some tasks where this product is identifiable but its factors are not. On those tasks the correct answer is a stated ambiguity, not a point estimate.
We cross two factors. The naming factor has three levels. In the canonical level, variables are called aerodynamic_resistance and evaporative_fraction and the site is described as, for example, "irrigated park." In the obfuscated level, identifiers are random strings and the site description is removed, while units and the simulator stay the same. In the misleading level, names and descriptions are coherent but wrong for the true parameters; for instance, a "dense forest" whose true evaporative fraction is that of dry pavement, mimicking degraded canopy (Zhu et al.). The parameter factor has two levels: values drawn from ranges typical for the described surface, or values outside them. This 3×2 design separates what the names contribute from what the feedback contributes.
For real-data grounding, we fit per-scene effective $\theta$ on LSTAT-20K, which pairs Landsat surface temperature with UrbClim 2 m air temperature across 100 European cities. Radiative forcing comes from ERA5, the gridded reanalysis used in the Iberian heat forecasting study. Ground truth for $\theta$ is unknown here, so we evaluate the air temperature predicted for held-out cities after mapping spectral indices (vegetation, built-up and water indices) to $\theta$. We also check a physical consistency criterion: for high- versus low-vegetation contrasts, the ratio of air-temperature change to surface-temperature change implied by the fitted parameters should fall near the observed 15% to 30%. One caveat: UrbClim is itself a boundary-layer model, so agreement partly measures agreement between models. We report it as such.
Baselines
The naive reference sets every parameter to the midpoint of its physical range and runs the forward model. On real data, it is complemented by a per-pixel linear regression from surface temperature and spectral indices to air temperature. This is the best classical baseline DiffTemp reports on LSTAT-20K, though the file gives no numerical RMSE for it. Any estimator that fails to beat the midpoint on parameter error has learned nothing from the data.
The name-invariant optimizers are Bayesian optimization and CMA-ES with uniform priors over physical ranges. They are the strongest competitors on the obfuscated conditions because they are, by construction, unaffected by names. The only reported comparison against them is LLMPhy's 6% to 12% accuracy advantage on rigid-body data; no reported result was found on Lacuna for any method on thermodynamic parameter identification. All optimizers and LLM methods get the same forward model, the same noisy observations, and the same budget of simulator calls (we plan 100 and 300). LLM methods also get a matched cap on tokens.
The LLM loop is our reimplementation of the LLMPhy procedure: the LLM writes parameter values, the simulator returns an error summary as text, and the LLM revises. We run it with an open model (Qwen2.5 family) and a reasoning model (DeepSeek R1 or o1). It counts as beating the optimizers only if its advantage in normalized parameter error survives the obfuscated condition with non-overlapping confidence intervals across at least 200 tasks.
The ablation that isolates our key question is LLM-prior Bayesian optimization. We ask the LLM once, with the same task description, for a mean and range for each parameter, then run Bayesian optimization from that prior with no further LLM calls. If the LLM loop fails to beat this ablation in the canonical condition, the loop's iterative reasoning adds nothing beyond a lexical prior. A second ablation strips the LLM loop's feedback down to a single scalar error, which tests whether it uses the structure in the feedback text.
An amortized neural estimator is trained on simulated tasks to map observations directly to $\theta$. It is name-invariant and fast. On real data, DiffTemp is the reference black box, with a reported RMSE of 2.20°C and SSIM of 0.70 on LSTAT-20K (Urban Air Temperature Prediction using Conditional Diffusion Models). UrbanHeat's reported MAE of 0.512°C comes from Seoul weather stations and cannot be compared directly. The physical model does not need to beat DiffTemp's RMSE. It needs to come within about 0.5°C while exposing parameters that pass the 15% to 30% consistency check, which DiffTemp cannot provide.
Why not just…?
Why not just use Bayesian optimization or CMA-ES and drop the LLM? This is a serious alternative, and the proposal is built so that it may win. Three parameters are a small space, and LLMPhy's reported margin over these optimizers is modest. If the optimizers match the LLM loop on obfuscated tasks and LLM-prior Bayesian optimization matches it on canonical tasks, we will report that the practical recipe is "elicit once, then optimize." That is a useful negative result for the whole LLM-in-the-loop physics line.
Why not just train an amortized neural network on simulated data? In-distribution, it will probably be the fastest and most accurate method, and we expect it to win there. Its known weakness is parameter shift: ExPhy found that learned estimators degrade outside training ranges, and the off-prior condition tests exactly this. We train it on the canonical ranges only. If it matches the loop and the optimizers on off-prior tasks as well, it dominates, and we will say so.
Why not just use DiffTemp or UrbanHeat for the air temperature and skip parameters altogether? For mapping air temperature in familiar cities, the black boxes are hard to beat. They cannot, however, answer the questions that motivate the surface-to-air literature: how much of an observed surface cooling reaches the air, and why. The LSTAT-20K experiment measures what accuracy is lost in exchange for that interpretability.
Why not just prompt with chain-of-thought and let the model self-correct? The LLM-Modulo evidence shows that self-critique without an external verifier degrades answers. We include a no-simulator chain-of-thought condition only as a lower bound, and we expect it to be the worst LLM variant.
Proposed Experiments
The first decisive check is the obfuscation gap. For each estimator, we compute the difference in normalized parameter error between the canonical and obfuscated conditions. The optimizers and the amortized network should show a gap of zero. We predict a large gap for the LLM loop, in the spirit of the drop from 34.3% to 2% on PlanBench.
The second check is the prior-equivalence test. In the canonical, on-prior cell, we compare the LLM loop to LLM-prior Bayesian optimization at matched call budgets. Statistical equivalence within a margin of 0.02 in normalized error supports the lexical-prior hypothesis. A clear loop advantage refutes it.
The third check is the misleading-label test. We predict that the LLM loop and the LLM-prior optimizer do worse than uniform-prior optimization when labels mislead, and that the loop anchors on the stated surface type even when the feedback contradicts it. We will measure anchoring as the number of calls needed to leave the label-implied range.
The fourth check is ambiguity detection. On the non-identifiable tasks, we score whether each method reports the trade-off between $r_a$ and $\epsilon$, either as a verbal statement from LLM methods or as a posterior ridge from Bayesian optimization.
Finally, if the obfuscation gap is large, we test a remedy. Following CAPT, which reached 83.75% on anti-sense PrOntoQA from 100 training examples with symbol randomization (Causality-Aware Post-Training), we fine-tune a small model on obfuscated parameter-inference episodes. We then ask whether it closes the gap without losing canonical performance. Success would show that feedback-driven search can be taught separately from vocabulary. Failure would suggest that, for continuous inverse problems, LLMs should supply priors and leave the search to optimizers.
References
- Observed different impacts of potential tree restoration on local surface and air temperature
- LLMPhy: Parameter-Identifiable Physical Reasoning Combining Large Language Models and Physics Engines
- DiffWind: Physics-Informed Differentiable Modeling of Wind-Driven Object Dynamics
- Large Language Models Still Can't Plan (A Benchmark for LLMs on Planning and Reasoning about Change)
- Lexical Recall or Logical Reasoning: Probing the Limits of Reasoning Abilities in Large Language Models
- LLMs Still Can't Plan; Can LRMs? A Preliminary Evaluation of OpenAI's o1 on PlanBench
- Mitigating Spurious Correlations in LLMs via Causality-Aware Post-Training
- Position: LLMs Can’t Plan, But Can Help Planning in LLM-Modulo Frameworks
- Measuring Fine-Grained Urban Air Temperature with Satellite Imagery
- Urban Air Temperature Prediction using Conditional Diffusion Models
- ExPhy: A Benchmark for Explicit Physical Property Learning in Multi-Object Trajectory Forecasting
- Comparable biophysical and biogeochemical feedbacks on warming from tropical moist forest degradation
- A Lightweight Deep Learning Approach to Spatiotemporal Heat Forecasting