Shear-Thinning Isomorphs: Using Non-Newtonian Flow to Expose Memorized Physics Reasoning

When a language model solves a fluid-mechanics problem, we usually cannot tell whether it reasoned from the governing equations or recalled a textbook answer it saw during training. The standard textbook case is Newtonian flow, meaning fluids like water whose thickness (viscosity) stays constant however fast they are sheared. That case is everywhere on the web. Non-Newtonian fluids are different: blood, batter, sauces and polymer melts thin or thicken as they are sheared, and their answers depart from the textbook ones in ways that are exactly computable. This proposal builds a problem generator around that fact. It produces fresh fluid-mechanics problems with a continuous knob, the rheology parameters, that moves a problem smoothly away from the memorized Newtonian answer while keeping the derivation the same and the ground truth exactly known.

The core claim has two parts. First, pairs of problems that share a derivation but differ in constitutive law and numbers (we call them structural isomorphs) can reveal whether a model's benchmark score has been inflated by exposure to the test items. The signal is the gap between its score on a published item and its score on that item's fresh isomorphs. Second, non-Newtonian rheology adds a diagnostic that resampling numbers alone cannot give. A model that is recalling rather than deriving should err toward the Newtonian answer, and we can measure that pull directly. We will test whether this audit still works when standard contamination detectors are known to fail, namely after reinforcement-learning post-training.

Why this gap, and what it builds on

The contamination literature supplies the motivation. Benchmark Data Contamination of Large Language Models: A Survey documents contamination rates from 1% to 45% on public QA benchmarks and large score drops on rewritten or post-cutoff problems. DyVal generates reasoning problems on the fly from directed acyclic graphs and shows that models strong on static tests collapse on fresh ones. Its generators, however, are purely discrete: arithmetic, Boolean logic and graph search. We take its central move, generating rather than publishing, and bring it to continuous physics, where every problem has a physical parameter controlling how far it sits from well-known cases.

Two negative results constrain the design. On The Fragility of Benchmark Contamination Detection in Reasoning Models shows that one epoch of clean GRPO post-training pushes detectors toward chance. GRPO is a reinforcement-learning method whose clipped objective shrinks the log-probability gap between seen and unseen items. The same paper finds that contamination with chain-of-thought data evades all ten detectors it tested, which averaged 50–58% AUROC. Any audit that reads token probabilities is therefore suspect, so ours is purely behavioral. The Emperor's New Clothes in Benchmarking? shows that surface rewrites keep fidelity above 0.9 but give almost no contamination resistance, while aggressive rewrites drop MMLU fidelity to 0.686. Fidelity here means that clean models score the same on original and modified items. Our isomorphs are meant to escape this trade-off. Because the derivation is held fixed and the answer is recomputed exactly, the problem changes without its difficulty drifting in unknown ways.

The fluid side supplies the physics and the traps. Learning Unknown Physics of non-Newtonian Fluids works in one-dimensional channel flow. It learns the viscosity–shear-rate law with physics-informed networks and highlights that the power-law model predicts unphysical infinite viscosity at the channel centre, where shear is zero. That singularity makes a clean probe: a model that pattern-matches will report a finite number, while a model that reasons about the stated law will not. Blunted, non-parabolic velocity profiles in blood (Particle Image Velocimetry of the Blood Flow in a Micro-channel Using the Confocal Laser Scanning Microscope) give a real-world template for questions where the Newtonian answer is qualitatively wrong. The robotic dispensing work (Kitchen Artist) spans culinary liquids from 1 to 70,000 cP, and we use that range to keep generated parameters physically plausible.

The closest existing work is ContPhy, a video benchmark of continuum physics that includes liquids. It is static and template-generated, and its fluids differ only in density and gross viscosity, with no shear-dependent laws. It asks whether models understand physics. It does not ask whether a score can be trusted.

Research Program

We will build a generator we call RheoGen. Each problem template fixes a geometry and a question type. Geometries include pressure-driven channel and pipe flow, gravity-driven film flow on an incline, and plug flow of a fluid with a yield stress. Question types include flow rate, wall shear stress, plug width, and velocity or viscosity at a stated position. The generator samples a constitutive law from a family (Newtonian, power-law, Bingham, Carreau) along with its parameters and the boundary data. Ground truth comes from closed-form solutions where they exist and from a one-dimensional quadrature or shooting solver otherwise, so each answer is exact to solver tolerance. A difficulty coordinate $d$ measures distance from the Newtonian case, for example $|1-n|$ for power-law index $n$ or yield stress normalized by wall stress. This gives a curve analogous to the phase-transition sweep in Can Large Language Models Reason? A Characterization via 3-SAT.

For each numeric question we define a Newtonian anchoring index,

$$A = \frac{\hat{y} - y^{*}}{y_{N} - y^{*}},$$

where $\hat{y}$ is the model's answer, $y^{*}$ the true answer for the stated fluid, and $y_{N}$ the answer a Newtonian fluid of matched reference viscosity would give. $A\approx 0$ means correct, and $A\approx 1$ means the model substituted the textbook answer. For a published item $i$ with isomorph set $S_i$, the contamination gap is $\Delta_i = \mathrm{acc}(i) - \mathrm{acc}(S_i)$, averaged over the benchmark. Clean models should show $\Delta\approx 0$ and anchoring that rises smoothly with $d$. Contaminated models should show positive $\Delta$ concentrated on the published items.

Baselines

The naive reference point is to report accuracy on a fixed, published set of RheoGen items, as a static benchmark would. It is what leaderboards do today, and it is the number contamination inflates, so it defines the error we aim to correct.

The strongest existing detection competitor is LiRA, a reference-model likelihood-ratio membership test, as run in On The Fragility of Benchmark Contamination Detection in Reasoning Models. That paper reports 89.1% AUROC after supervised contamination, falling by about 14% after one GRPO epoch. We will reproduce that pipeline on the same open model family (DeepSeek-R1-Distill-Llama-8B) with the same number of GRPO steps, and substitute our fluid items for the math sets. LiRA gets the same reference models and query budget that our isomorph audit gets in generated samples. Beating it means our isomorph gap achieves higher AUROC than LiRA at separating contaminated from clean checkpoints after GRPO, where the fragility result predicts LiRA near chance. We will also report loss-threshold and Min-K% detectors from the same paper.

The strongest mitigation competitor is paraphrasing, following The Emperor's New Clothes in Benchmarking?, which reports fidelity above 0.9 and near-zero resistance. We will compute its fidelity and resistance on our items with the paper's definitions and place isomorphs on the same plot. Beating it means matching its fidelity with clearly higher resistance.

The key ablation isolates the rheology knob. Here isomorphs resample only numbers and keep the fluid Newtonian, which is DyVal-style regeneration applied to physics. If this ablation recovers the same $\Delta$ and the same detection AUROC as full isomorphs, the non-Newtonian ingredient adds nothing for detection. It would then only matter for the anchoring diagnostic.

For external reference on the fluid side, the static ContPhy fluid-property results are GPT-4V at 25.0% and the simulator-coupled ContPRO at 78.0%. We cite these for scale only, since the modality and task differ.

Why not just…?

Why not just resample the numbers in Newtonian problems, as DyVal does for arithmetic? This is the real threat. Number resampling may defeat verbatim recall about as well as our isomorphs do. The ablation above settles it. Our prediction is that it catches answer-level memorization but misses memorized solution templates: a model trained on contaminated worked solutions can reuse the Newtonian derivation with new numbers. Only a changed constitutive law, with its anchoring signature, exposes that. If the prediction fails, we will report RheoGen's value as diagnostic rather than as a detector.

Why not just paraphrase existing fluid questions? The mitigation study already shows that surface edits leave contaminated models free to retrieve memorized answers. We will confirm this on our items, not assume it.

Why not just run a membership detector? The fragility paper shows that GRPO drives these detectors toward chance and that chain-of-thought contamination evades all ten it tested. Our behavioral test never reads log-probabilities, which is precisely why it should survive that post-training step.

Why not just let the model call a solver? The 3-SAT study shows that translating to a symbolic solver gives near-perfect accuracy, and ContPRO shows a similar pattern. That measures translation, not physical reasoning. We will run a tool-use condition anyway. If the isomorph gap vanishes there, that result is useful in its own right: tool use would then be a partial defense against contamination.

Proposed Experiments

We first validate the generator by checking solver answers against closed forms and by having domain experts verify a sample of problems for well-posedness. We then run the controlled contamination study. Clean and contaminated checkpoints are trained, with contamination introduced by supervised fine-tuning on published items, either answers only or full worked solutions, and each checkpoint is then post-trained with GRPO. We measure $\Delta$, anchoring curves $A(d)$ and detector AUROC at each stage. Finally we evaluate frontier and open models on fresh RheoGen draws to chart anchoring against distance from Newtonian behavior.

The decisive checks are three. After GRPO, the isomorph gap must separate contaminated from clean checkpoints where LiRA does not. Solution-contaminated models must show anchoring near 1 on high-$d$ isomorphs that the Newtonian-only ablation misses. Clean models must show $\Delta$ statistically indistinguishable from zero across resamples, which is our fidelity check. If the first check fails, the proposal fails. If only the second fails, the non-Newtonian knob is decorative, and we will say so.