Inverse rendering recovers a 3D object's shape, surface materials and lighting from photographs. When a method simulates fewer light bounces than actually occurred, the optimizer has to explain the missing light somehow, and the cheapest explanation is to brighten or tint the surface color, also called the albedo: the color a surface would have under neutral white light. The resulting "baked-in" lighting artifacts are what make a reconstructed asset look wrong when it is moved under new lights. The fast pipelines in current use stop the series of bounces after one indirect bounce. These include deferred shading on Gaussian splats (GI-GS), screen-space reflections (3D Gaussian Inverse Rendering with Approximated Global Illumination) and direct-only mesh pipelines (Extracting Triangular 3D Models, Materials, and Lighting From Images). Simulating every bounce with Monte Carlo path tracing, which estimates light by averaging many random ray paths, is too slow and too noisy to run inside the optimization loop.
Device physicists hit the same structural problem when simulating electrons scattering off lattice vibrations, and they solved it without random sampling. They computed only the first few terms of the scattering series, the "lowest order approximation" or LOA. They then replaced the cut-off sum with a Padé approximant: a ratio of two short polynomials whose coefficients are fitted to those terms, so that it stands in for the infinite remainder. This recovered more than 90% of the accuracy of the full self-consistent solution at roughly six times less cost (Quantum Treatment of Inelastic Interactions for the Modeling of Nanowire Field-Effect Transistors). We propose to do the same for light. At each surface point, we compute the direct term and the first two indirect bounces cheaply. A per-point Padé formula estimates every remaining bounce. The estimate stays a differentiable function of the current material estimate, so the optimizer no longer gains anything by inflating the albedo.
Core Claim
Cutting the bounce series off introduces a bias that is systematic, not noisy, and it grows with how strongly a surface sees itself. Flash Cache showed that systematic bias does not average out during optimization. Instead it steers the gradients toward wrong solutions and burns shading into the albedo. Its remedy is an unbiased Monte Carlo estimator with a control variate, which costs extra variance and a second radiance representation.
Our claim is that most of this bias can be removed deterministically. Write the radiance leaving a point $x$ as a series in a bookkeeping parameter $\lambda$ that multiplies the light transport operator:
$$L(x;\lambda)=\sum_{k\ge 0}\lambda^k L_k(x),$$
where $L_0$ is direct lighting and $L_k$ is light that has bounced $k$ times between surfaces. The $[1/1]$ Padé approximant built from $L_0, L_1, L_2$ and evaluated at $\lambda=1$ is
$$\hat L(x)=L_0(x)+\frac{L_1(x)}{1-r(x)},\qquad r(x)=\min\!\left(\frac{L_2(x)}{L_1(x)},\ \bar\rho(x)\right),$$
computed separately for each color channel. Here $r$ is the locally estimated ratio between successive bounces. Its cap $\bar\rho(x)$ is the directional albedo of the current material, meaning the fraction of incoming energy the surface can reflect at all. The cap is an energy-conservation bound. It plays the role that the current-conserving rescaling played in the transport work (Quantum treatment of phonon scattering for modeling of three-dimensional atomistic transport): it keeps the resummed tail physical without bringing back the iterative loop.
We want to be upfront that this formula is classical. It is the Shanks transform of a series of nonnegative terms. The contribution lies elsewhere, in three places:
- placing the formula inside differentiable inverse rendering, so the estimated tail depends on the albedo and roughness being optimized;
- bounding the tail by the material's own energy budget;
- showing whether doing so removes albedo inflation in exactly the regime where truncation hurts most.
That regime is concavities, bowls, folds and metallic interiors whose surface reflectance approaches one. There, the one-bounce truncation misses a fraction of indirect light of about $r/(1-r)$, which comes from summing the geometric tail. This gives a falsifiable prediction: albedo error in truncated baselines should grow with $r/(1-r)$ measured from ground truth, and our method should flatten that curve.
The light problem is actually better behaved than the electron problem. All bounce terms are nonnegative, so the Padé denominator cannot change sign unless $r\to 1$, and the cap rules that out. The transport method's documented failure on p-type ⟨111⟩ silicon came from scattering that increased current, a sign anomaly with no analog here. The real failure mode for light is a noisy or wrong $L_2/L_1$ ratio. We test for it directly.
Research Program
We build on GI-GS, because it already computes a differentiable one-bounce indirect term. Its approach is to render the color of the surface hit by a secondary ray from a buffer of per-pixel geometry and materials (the G-buffer), with a cubemap fallback for hits outside that buffer. The only new expensive quantity is $L_2$. We obtain it by reusing the one-bounce indirect value already computed at the secondary hit point, read from the same buffers. This adds one lookup per secondary ray and no new rays.
Gradients flow through $L_1$, $L_2$ and the cap $\bar\rho$. As a result, raising the albedo $a$ at a point in a cavity raises the predicted tail superlinearly, roughly as $a/(1-a\,F)$ where $F$ is how much the surface sees itself. Truncated models lack exactly this coupling.
Glossy and metallic surfaces are where existing fast methods give up (Physics-based Indirect Illumination for Inverse Rendering, TensoIR). For them we evaluate $r$ separately for diffuse and specular lobes and cap each lobe by its own lobe albedo, which drops the fixed-dielectric assumption of $F_0=0.04$ used in prior work.
The analysis component has two parts. First, we measure on synthetic scenes how accurate the Padé tail is relative to ground-truth path-traced bounce decompositions, as a function of $r$. Second, we measure the bias left in the albedo gradient, meaning the difference between the expected gradient under our estimator and under full transport. Following Flash Cache, this residual gradient bias is the quantity that should predict baking.
Baselines
The naive reference point is the same GI-GS pipeline with its native one-bounce truncation. It isolates exactly what resummation adds, at identical geometry, data and training schedule. On the TensoIR synthetic set, GI-GS reports novel-view PSNR of 36.75 against 35.33 for GS-IR (GI-GS). We found no reported GI-GS albedo PSNR on Lacuna, so we rerun it ourselves on every benchmark.
The strongest existing competitor on the specific question of bias from approximate indirect light is Flash Cache, the unbiased control-variate estimator. It reports Normal MAE of 3.355 against 4.100 for TensoIR on synthetic scenes. We give it the same wall-clock budget as our method and also its native budget, so that we can report both accuracy at equal cost and the cost needed to match accuracy. Beating Flash Cache means matching its albedo PSNR within noise at substantially lower training time, or exceeding it at equal time on the high-albedo cavity set.
For interreflection-heavy real scenes, the reference is the DIP and PII method (DIP). It reports albedo PSNR of 26.43 on the MII Air Balloons scene, versus 22.53 for the prior best. For real-world relighting, the anchor is NVDiffRecMC's PSNR-H of 24.43 on Stanford-ORB.
Four ablations isolate the key ingredient:
- Explicit two bounces. Two indirect bounces with no tail estimate, which tests whether resummation adds anything beyond one more term.
- Detached tail. The same Padé tail with gradients stopped through the materials, which tests whether the material-dependent coupling, rather than the extra energy itself, is what prevents albedo inflation.
- Uncapped tail. The tail without the energy cap, which tests the conservation constraint.
- Learned ambient term. A learned per-point constant added to the indirect light, which absorbs missing energy with no physics.
All variants share seeds, iteration counts and hyperparameter search budgets.
Why not just…?
Why not just trace a second explicit bounce? Two bounces cost about the same as our method, since our $L_2$ is exactly that second bounce. In a cavity with $r\approx 0.8$, however, stopping after two bounces still misses about 64% of the indirect energy, because $r^2/(1-r)\,/\,(1/(1-r)) = r^2$. The two-bounce ablation measures this directly. If it matches our method on the cavity set, resummation has no value, and we would say so.
Why not use the radiance field itself as the secondary light source, as TensoIR and the CVPR 2022 indirect-illumination method do? This is a genuine threat. A radiance field fitted to the photos already contains all bounces at any point the cameras observed. That is why Modeling Indirect Illumination for Inverse Rendering reaches an aligned albedo PSNR of 25.25 against NeRFactor's 22.96. The approach breaks down in two ways. First, the radiance field is unreliable inside rarely observed concavities. Second, being a smooth cache, it carries the low-frequency bias that Flash Cache identified. It also decouples indirect light from the materials being optimized. We will run it as a baseline and report results separately for well-observed regions and for occluded cavity regions, where we expect it to lose.
Why not amortize the full self-consistent loop across training steps by reusing last step's outgoing radiance as the bounce source? This is the inverse-rendering analog of running the self-consistent iteration slowly. It is also the closest simple competitor, because over many steps it accumulates all bounces. Its weakness is staleness: whenever materials change, the stored radiance lags behind them. We will implement it and compare convergence curves. If it matches our method at equal memory, the honest conclusion would be that Padé mainly speeds up this fixed-point iteration. In that case we would combine the two, in the way Richardson extrapolation sped up the transport iterations from 35 to 6.
Why not use unbiased Monte Carlo with many samples? It is correct in the limit, but it is the cost regime this work exists to avoid. NeRFactor-style pipelines needed over 100 hours per scene (TensoIR). We use it only to generate ground truth and as the Flash Cache comparison at matched time.
Proposed Experiments
We evaluate on four sets of scenes:
- the TensoIR synthetic scenes, for albedo PSNR, normal error and relighting;
- the MII scenes, for strong interreflections;
- Stanford-ORB, for real-world relighting (PSNR-H) and novel views, with emphasis on the pitcher and other glossy objects;
- a new synthetic cavity set that we render ourselves with a path tracer, using exact per-bounce decompositions. It contains white interiors, nested bowls and brushed-metal cups with surface reflectance from 0.3 to 0.95.
The decisive check runs on the cavity set. We plot albedo error against the ground-truth $r/(1-r)$ for the one-bounce baseline, the two-bounce baseline, the detached tail and our method. Our hypothesis predicts three things:
- the baselines' error rises steeply with $r/(1-r)$ while ours stays flat;
- the detached-tail ablation recovers the extra energy but still inflates albedo;
- relighting under held-out environment maps improves most where $r$ is high.
The second check measures how closely the Padé estimate tracks the true sum of all bounces, separated by lobe, and reports where the cap activates. This is to verify that the cap is a rare safeguard and not a quiet rescaling of the light. The third check is cost: training time and memory relative to GI-GS and Flash Cache.
If the Padé tail does not beat both the two-bounce and lagged-iteration baselines on the cavity set, the idea is falsified.
References
- GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering
- 3D Gaussian Inverse Rendering with Approximated Global Illumination
- Extracting Triangular 3D Models, Materials, and Lighting From Images
- Quantum Treatment of Inelastic Interactions for the Modeling of Nanowire Field-Effect Transistors
- Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering
- Quantum treatment of phonon scattering for modeling of three-dimensional atomistic transport
- Physics-based Indirect Illumination for Inverse Rendering
- TensoIR: Tensorial Inverse Rendering
- DIP: Differentiable Interreflection-aware Physics-based Inverse Rendering
- Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmark
- Modeling Indirect Illumination for Inverse Rendering