Inverse rendering works backward from photographs to the shape and material of an object. To do this, it simulates how light reflects and adjusts the scene until the simulation matches the photos. Most such systems assume every surface is a dielectric, an insulator like plastic or ceramic. Metals break that assumption. Their reflection depends on a complex refractive index $\tilde n = n + i\kappa$, where $\kappa$ (the extinction coefficient) is zero for insulators and positive for metals. That extra term also turns reflected light elliptically polarized. NeISF++ handles both material classes in one polarized reflectance model and cuts surface-normal error on mixed scenes from 10.303° (the dielectric-only NeISF) to 1.789°. It does this only with a manually supplied mask saying which pixels are metal.
This proposal asks a question neither field has posed. When that mask is removed and the optimizer must decide metal versus insulator on its own, how much does the answer depend on how the reflection code was compiled? Compiler research has shown that optimization flags reshape programs so much that tools reasoning about them degrade sharply. The conductor case of the Fresnel equations, the formulas for how much light a surface reflects, contains exactly the operations those flags rewrite: complex square roots, differences of nearly equal terms near grazing angles, and reciprocals. We hypothesize that "fast-math" rewrites introduce systematic, not random, errors in the circular-polarization and phase terms that separate metals from insulators. We further hypothesize that the ill-posed metal/insulator decision amplifies these errors into flipped material labels and biased $\kappa$ estimates. We will measure this, localize it to specific operations, and test whether compiling only those operations under strict arithmetic removes the effect at little speed cost.
Core Claim
The claim is that compiler numerical choices are a hidden experimental variable in mask-free conductor–dielectric inverse rendering. Their effect on recovered material labels can be comparable to seed-to-seed variation, and it concentrates in a small, predictable set of operations in the conductor branch of the reflectance model.
The claim has three parts, each falsifiable:
- The flag-induced effect is measurable.
- The effect is systematic rather than zero-mean.
- Its location can be predicted from each term's local numerical conditioning before any inverse run.
If fast-math changes nothing beyond seed noise, that is a useful negative result. It would license the fast configurations the field already uses.
Relation to Prior Work
The motivation comes from compiler work showing optimization is not semantically innocent for downstream reasoning.
- Can Large Language Models Understand Intermediate Representations in Compilers? sweeps -O0 to -O3. It finds GPT-4's execution-reasoning pass rate falls from 72% on source code to 36% on compiler intermediate representation.
- Tan et al. documents how arithmetic substitution hides algebraic intent.
- Decompile-Bench supplies a methodological habit we borrow: sweep optimization levels systematically and treat them as a controlled axis rather than a fixed setting.
All three study how optimization obscures program structure for an analyst. We instead study how it perturbs numerical outputs that feed an ill-posed optimization. That is a different failure mode, applied to kernels none of them evaluate.
Nearer graphics work treats compiled shading code as a fixed black box.
- The neural deferred shaders (A Convolutional Neural Deferred Shader for Physics Based Rendering, Beyond Reconstruction) approximate shading with networks. They do not ask how shader compilation affects losses. - Identifying shader sub-patterns for GPU performance tuning profiles compiled shaders for throughput only.
We keep their concern with compiled execution but measure physical correctness instead of speed.
On the optics side, NeISF++ supplies the reflectance model, the benchmark, and the DoLP-based geometry initialization. DoLP, the degree of linear polarization, is $\sqrt{Q^2+U^2}/I$ for Stokes vector $[I,Q,U,V]$. NeISF++ also constrains the design. Its manual mask is exactly what hides the ill-posedness we want to expose, so our main setting replaces that mask with a learned per-point conductor probability. PhySG and Tao et al. motivate why this decision matters: dielectric-only models misread colored metallic highlights as albedo or geometry.
Research Program
Kernel. We will reimplement the NeISF++ unified polarized reflectance evaluation, both its forward and backward passes, as a standalone C++/CUDA kernel called from the existing pipeline. This lets us change compilation without touching anything else.
Flag matrix. Compile the kernel under the following configurations:
- a float64 reference;
- strict IEEE float32;
- full fast-math;
- each fast-math component alone: reassociation, approximate reciprocal and square root, flush-to-zero of tiny values, fused multiply-add contraction, and assume-no-NaN;
- GPU intrinsic approximations for square root and arctangent.
Forward analysis. On dense grids of incidence angle, $n$, and $\kappa$, we record the error of each Stokes component against float64. Our prediction is that the error is largest in $V$ and in the polarization phase near grazing angles for large $\kappa$.
Separately, we estimate each intermediate operation's relative condition number by finite differences in float64. This predicts where flag errors should matter, before seeing any inverse result. We also count physically impossible outputs, such as $I < \sqrt{Q^2+U^2+V^2}$ or DoLP greater than 1. These are the validity checks raised for compiled neural fields in the research file, applied here to the analytic kernel.
Inverse analysis. We run mask-free and masked inverse rendering under every configuration, with multiple random seeds per configuration. We summarize the effect of compilation on any recovered quantity $y$ with
$$S_y = \frac{\operatorname{Var}_{\text{flags}}\big(\bar y_f\big)}{\operatorname{Var}_{\text{seeds}}(y)}.$$
Here $\bar y_f$ is the seed-averaged outcome under flag configuration $f$. The denominator is the pooled within-configuration variance across seeds. $S_y \gg 1$ means compilation matters beyond optimizer randomness.
Selective strict compilation. Finally, we compile only the operations flagged by the conditioning analysis under strict arithmetic and leave everything else fast. We then measure accuracy and runtime.
Baselines
Naive reference. The naive reference is NeISF++ as released, with its default compilation and its manual mask. Its reported normal mean angular error on the synthetic conductor–dielectric benchmark is 1.789° (NeISF++). We rerun it on the same scenes to confirm we reproduce roughly that figure before changing anything. The dielectric-only NeISF, reported at 10.303° on the same benchmark, anchors the cost of ignoring metals entirely.
Strongest competitor: the float64 kernel. This is the obvious fix and the correctness ceiling. All configurations share identical scenes, iteration counts, learning-rate schedules, and seed sets. We report wall-clock time alongside accuracy.
Practical competitor: all-strict float32. This is simply turning fast-math off. Selective compilation "beats" it only if it matches its accuracy within seed noise at a measurably lower runtime. If there is no runtime gap, we will say that strict float32 is the answer.
Ablation isolating the key ingredient. Selective strict compilation is guided by the conditioning analysis. We compare it against two controls:
- selection of the same number of operations at random;
- selection of only the dielectric-branch operations.
This tests whether the prediction, not just the reduced amount of fast arithmetic, carries the benefit.
Sensor-noise reference. We inject a polarization-camera-like noise model into the synthetic images. This measures the ratio of flag-induced to noise-induced error.
No reported result found on Lacuna for mask-free NeISF++, or for any compiler-flag sensitivity of polarimetric inverse rendering.
Why not just…?
Why not just compile everything in float64 or disable fast-math and move on? This is the most likely practical outcome, and the program is built to test it rather than avoid it. If strict float32 closes the gap at negligible cost, the contribution reduces to a measurement: the field's fast configurations bias mask-free material recovery, plus a default recommendation. The selective scheme is only worth proposing if the runtime comparison against all-strict float32 shows a real gap. That comparison is a reported result, not an assumption.
Why not just assume sensor noise swamps floating-point error? This is a genuine threat. Polarization sensors have low signal-to-noise, and it is plausible that compiler error is irrelevant next to photon noise. Our counter-hypothesis is that fast-math errors are deterministic functions of angle and $\kappa$. They should therefore bias every view in the same direction, while sensor noise averages out across views. The noise-injection baseline settles this. The test is whether recovered $\kappa$ and mask labels under fast-math stay biased as the number of views grows, while noise-induced error shrinks.
Why not just average over random seeds? GPU nondeterminism already produces run-to-run variation, and averaging over seeds handles that. The ratio $S_y$ is defined to separate the two sources. If it stays near 1, seed averaging is sufficient and our claim fails.
Why not just keep the manual mask? With the mask supplied, the conductor branch never competes with the dielectric branch. Any sensitivity would then show up only as small $\kappa$ errors. That is why we run masked and mask-free variants side by side. Automating the mask is the open problem NeISF++ names, and we expect compiler sensitivity to bite hardest precisely when the mask is removed.
Proposed Experiments
Data. All experiments use the synthetic Mitsuba-rendered conductor–dielectric benchmark from NeISF++ and the NeISF teapot and bunny scenes. The real FLIR polarimetric captures are used for qualitative checks only, since no numerical ground truth is reported for them.
Metrics.
- surface-normal mean angular error;
- error in recovered $n$ and $\kappa$;
- conductor-mask agreement (intersection over union against the ground-truth mask);
- rate of physically invalid Stokes outputs;
- runtime per iteration.
Decisive checks.
- Existence. $S_y$ for mask agreement and $\kappa$ exceeds 1 by a margin that survives a permutation test over flag labels.
- Direction. Fast-math-induced $\kappa$ bias has a consistent sign across seeds and does not shrink as the number of views grows. The sensor-noise-induced error should shrink under the same change.
- Prediction. The operations ranked worst by the conditioning analysis account for most of the inverse-rendering degradation. If they fix the problem where randomly chosen operations do not, the predictive claim holds.
A null result on the first check ends the project with a clean negative finding, which we would publish as such.
References
- NeISF++: Neural Incident Stokes Field for Polarized Inverse Rendering of Conductors and Dielectrics
- NeISF: Neural Incident Stokes Field for Geometry and Material Estimation
- Can Large Language Models Understand Intermediate Representations in Compilers?
- SK2Decompile: LLM-based Two-Phase Binary Decompilation from Skeleton to Skin
- Decompile-Bench: Million-Scale Binary-Source Function Pairs for Real-World Binary Decompilation
- A Convolutional Neural Deferred Shader for Physics Based Rendering
- Beyond Reconstruction: A Physics Based Neural Deferred Shader for Photo-Realistic Rendering
- Identifying shader sub-patterns for GPU performance tuning and architecture design
- PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting
- Depth Estimation for Glossy Surfaces with Light-Field Cameras