Sparse Drivers: Spatially Adaptive Smoothing in Factor Coordinates for Exact Emission Decompositions

Emission accountants routinely split a change in total emissions into contributions from population, wealth, energy use per unit of output, and carbon per unit of energy. The tool they use is the Logarithmic Mean Divisia Index (LMDI), an exact accounting formula whose pieces sum to the total change with nothing left over. In practice it runs on province or city statistical yearbooks, one administrative unit at a time. That hides what happens inside units and gives no error bars. Fine-grained data such as satellite land-cover grids or census tracts could expose local drivers, but those data are noisy, and ordinary smoothing either blurs the sharp local features that matter (a new industrial park, a policy border) or fails to remove the noise.

This proposal changes one thing: where the smoothing penalty is placed. We fit log-scale surfaces for each accounting factor on a spatial graph using a penalty from trend filtering. Trend filtering is a smoother that allows a few sharp jumps and keeps everything else flat. The key choice is to penalize jumps in the factors (for example carbon per unit of energy), not in the raw quantities or the total. Because the factors add up exactly in log space, the fitted surfaces reproduce the fitted total at every location by construction. The scientific bet is that drivers change sparsely across space and time even when the raw quantities do not. We test whether that bet yields more accurate local attributions than smoothing the raw quantities, with calibrated uncertainty.

Core Claim

The research file frames the gap as "identity versus smoothing": if one smooths total emissions and each factor separately, the product of smoothed factors no longer equals the smoothed total. That framing overstates the difficulty. Working in logarithms and defining the total as the sum of log-factors restores the identity trivially. The identity is not the hard part. The hard part is the choice of penalty coordinates. The ℓ1 penalty in trend filtering does not survive a change of variables. Penalizing jumps in log population, log GDP, log energy and log carbon separately is a different estimator from penalizing jumps in log affluence, log energy intensity and log carbon intensity, even though the two sets of coordinates are linearly interchangeable.

Our claim is that the factor-coordinate penalty matches how drivers behave. At a city edge, population and GDP jump together, so their ratio stays smooth. Carbon intensity, by contrast, may jump at a single power plant or a provincial policy line. Separate penalties on the raw quantities place their jumps at slightly different nodes. Taking ratios then produces spurious ridges in the factor maps, and those ridges flow straight into the attributions. The factor-coordinate penalty removes them.

Concretely, let $f_k(v,t)$ be the log of factor $k$ at graph node $v$ and time $t$, and let $y_m(v,t)$ be the observed log of raw quantity $m$. Each raw quantity is a known signed sum of factors, $u_m = \sum_k A_{mk} f_k$, with $A$ a fixed triangular 0/1 matrix given by the Kaya identity. We solve

$$\min_{f}\ \sum_{m,v,t} \rho\big(y_m(v,t) - (Af)_m(v,t)\big) + \sum_k \lambda_k \|D_s f_k\|_1 + \sum_k \mu_k \|D_t f_k\|_1,$$

where $\rho$ is squared or check loss, $D_s$ is a graph difference operator over space, $D_t$ is a difference operator over time, and $\lambda_k, \mu_k$ are per-factor penalty weights. LMDI is then applied node by node to $\exp(Af)$. It is exact because $\log V = \sum_k f_k$.

Research Program

The spatial penalty is taken directly from Trend Filtering on Graphs, which puts an ℓ1 penalty on graph differences of a single signal and preserved sharp urban peaks on Allegheny County tracts that Laplacian smoothing blurred. On regular satellite grids we use the separable operator of Multivariate Trend Filtering for Lattice Data. That paper supplies fast specialized ADMM solvers and a proof that linear smoothers can be inconsistent for signals of uneven smoothness, which motivates avoiding them. Both papers treat one response. Our contribution is the multi-signal version tied together by an accounting matrix, with the penalty moved into factor coordinates.

For heavy-tailed pixel noise we replace squared loss with the check loss used in Quantile Additive Trend Filtering. That paper shows least-squares additive trend filtering collapsing under Cauchy noise, with MSE above 1000 against below 1.0 for the quantile version. This result constrains the design: satellite-derived inputs should not be fit with squared loss.

The solver is ADMM with one splitting variable per factor-difference block. Each block's update is the known trend filtering proximal step, and the data step is a small linear solve because $A$ is triangular.

Uncertainty comes from a block bootstrap over spatial patches and time, refitting and recomputing the node-level LMDI terms. This answers the complaint in Personal GHG emissions accounting and the driving forces decomposition in the past 10 years that yearbook decompositions admit no standard errors.

On the theory side, each LMDI term equals the logarithmic mean of $V^T$ and $V^0$ times $f_k^T - f_k^0$, and the logarithmic mean is Lipschitz when $V$ is bounded away from zero. We therefore expect attribution error to inherit the trend filtering rate for each $f_k$. We will state this bound precisely and check it empirically rather than assume it.

We also add an optional aggregation constraint: node-level fitted raw quantities must sum to published yearbook totals. The fine maps then never contradict official accounts.

Baselines

The naive reference point is what the field does now: independent LMDI per unit on raw observations, as in Spatio-Temporal Dynamics, Driving Mechanisms, and Decoupling Evaluation of Farmland Carbon Emissions: A Case Study of Shandong Province, China across 16 prefectures, applied at whatever resolution the data allow. It uses no pooling and gives no error bars. Any method that cannot beat it on attribution error against known truth is not worth using.

The strongest competitor is primitive-coordinate trend filtering. This means graph trend filtering applied separately to each log raw quantity, with the same loss, the same solver, the same graph, and per-quantity penalties tuned by the same cross-validation budget (an equal number of penalty configurations on the same held-out nodes). Factors are then computed as ratios. It also preserves the identity exactly, so it isolates the one thing we change: penalty coordinates.

The ablation that isolates adaptivity is our factor-coordinate model with a squared Laplacian penalty in place of the ℓ1 penalty. A second ablation swaps check loss for squared loss.

We also include BAST from BAST: Bayesian Additive Regression Spanning Trees for Complex Constrained Domain, fit to the log total and to each factor separately. It is included as the strongest boundary-respecting adaptive regressor in the file. That paper reports Aral Sea chlorophyll MSPE of 2.346 for BAST versus 2.894 for soap-film smoothing and 2.933 for BART, but for single-response prediction.

No reported result was found on Lacuna for node-level factor attribution error for any method. We will therefore report all numbers ourselves on shared data. Beating the primitive-coordinate baseline means a lower attribution error summed over factors, with non-overlapping bootstrap intervals, on at least the simulated jump-in-intensity regime, and no significant loss elsewhere.

Why not just…?

Why not just run LMDI on finer administrative units, such as counties? Finer units do not add error bars, and data coverage thins as units shrink. An Evaluation of Chinese Rapeseed Production Efficiency Based on Three-Stage DEA and Malmquist Index shows how aggregation hides local variation, but disaggregating the tables does not remove noise. Our semi-synthetic experiment measures directly how much raw per-unit LMDI error grows as noise per unit rises. Pooling should win exactly when counts per node are small.

Why not just smooth each raw quantity with graph trend filtering and take ratios? This is the real threat, because it is equally exact and uses the same machinery. If jumps in population, GDP, energy and carbon rarely co-occur, or if noise is small, it may perform as well as our method. The simulation settles this by varying how often raw quantities share jump locations. We predict our advantage grows with sharing and vanishes without it. If it never appears on the tract graph or satellite grids, the method reduces to a reparameterization note and we will say so.

Why not just smooth the total with an adaptive regressor like BAST and allocate change to factors afterwards? A black-box fit of the total says nothing about which factor moved. Fitting each factor separately with BAST breaks the identity: the product of separately fitted factors will not equal the fitted total, so decompositions stop summing. We measure that residual and report it alongside attribution accuracy.

Proposed Experiments

The first test is semi-synthetic, on the Allegheny County tract graph from Trend Filtering on Graphs. We plant known factor surfaces: smooth population, affluence with a shared city-edge jump in the underlying population and GDP, and carbon intensity with a few isolated tract-level jumps across two periods. We then generate raw quantities with Gaussian, Poisson and Cauchy noise. The primary metric is mean absolute error of node-level LMDI terms against the truth. Secondary metrics are bootstrap interval coverage at 90% and false jump detection in each factor.

The decisive check is the co-location sweep described above. The prediction is that our method's advantage over the primitive-coordinate baseline rises monotonically with the fraction of shared jumps.

The second test uses the global impervious-surface grids in Substantial terrestrial carbon emissions from global expansion of impervious surface area. That paper's identity (urban population, urbanization rate, land per person, carbon density of converted land) was applied only to national aggregates, and its inputs come from four disagreeing satellite products. We fit on one product and score each method by agreement of its factor maps with the pointwise median of the other three, a held-out proxy for truth. As a consistency check, we sum node-level losses and require that they fall within that paper's range of 46–75 Tg C per year.

The third test uses the Shandong prefecture panel. With only 16 nodes it is a stress test of the temporal penalty and of the aggregation constraint, not of spatial resolution. We check that fitted attributions reproduce the province totals reported by the study (a peak of 29.4 million tons in 2007 and 20.2 million tons in 2023) and its decoupling classifications. We also test whether the bootstrap intervals are wide enough to call the 2004–2013 prefecture-level decoupling differences uncertain.

A negative outcome on the co-location sweep would be the clearest reason to abandon the factor-coordinate penalty.