Will a perfect model agree with perfect observations? The impact of spatial sampling

Nick Schutgens, Edward Gryspeerdt, Natalie Weigum, Svetlana Tsyro, Daisuke Goto, Michael Schulz, Philip StierAtmospheric 2016 · DOI

Abstract

The spatial resolution of global climate models with interactive aerosol and the observations used to evaluate them is very different. Current models use grid spacings of ~200 km, while satellite observations use pixels of ~10 km. We study the errors incurred due to different resolutions by aggregating high-resolution simulations (10 km grid spacing) over either the large areas of global model grid boxes ("perfect" model data) or small areas corresponding to satellite pixels or ground sites ("perfect" observations). Our analysis suggests that instantaneous root-mean-square (RMS) differences can reach 30–160% for variables like AOT, black carbon mass, and PM2.5. These differences, due entirely to spatial sampling, are often larger than measurement errors in real observations.

Introduction: The Resolution Mismatch

In climate science, global aerosol models are evaluated by comparing outputs to observations from satellites, ground stations, and aircraft. However, a fundamental "apples-to-oranges" problem exists. A typical global climate model (GCM) divides the world into grid boxes roughly 200 km wide, calculating a single average value for that entire area. Conversely, a satellite might look at a 10 km pixel, and a ground station measures a single point. Because aerosol concentrations vary wildly over short distances, a single point rarely matches a 40,000 km² average. This study quantifies the "spatial sampling error"—the inherent disagreement caused purely by this resolution mismatch.

Methodology: The Synthetic Truth Approach

The authors employ a "synthetic truth" approach using high-resolution regional model simulations (10 km grid spacing) to represent the "true" state of the atmosphere. From this high-res truth, two datasets are generated:

  1. "Perfect" Model Data: High-resolution truth averaged over large areas (e.g., 210 x 210 km) to mimic a GCM grid box.
  2. "Perfect" Observations: A single 10 km pixel or point taken from within that same area.
Global distribution of high-resolution simulation domains used as 'synthetic truth' across various regions and modeling frameworks.

The study utilizes simulations from WRF-Chem, EMEP, and NICAM-SPRINTARS across six diverse regions, including Europe, Oklahoma, and the Congo, capturing various meteorological conditions and aerosol source types.

Quantifying the Sampling Error

Comparing these "perfect" datasets reveals that the spatial sampling error is substantial. Instantaneous differences between a perfect model and a perfect observation range from 30% to 160%. For many variables, this sampling noise exceeds the actual measurement error of the instruments.

Spatial distribution of aerosol optical thickness (AOT) and surface black carbon (BC) concentrations (left panels) and their corresponding relative spatial sampling errors (right panels).

Temporal averaging (e.g., monthly means) reduces error but does not eliminate it. While AOT errors might drop three-fold, surface black carbon errors only drop by half, leaving residual errors of 10–75%. Furthermore, the location within a grid box matters; observations at the corner of a model grid box can have double or triple the sampling error compared to the center.

Implications for Model Evaluation

The study highlights specific risks for flight campaigns. Aircraft often target pollution plumes, which can introduce a massive positive bias. This makes a model appear to underestimate aerosols when it might be correct for the grid-cell average.

To mitigate these errors, the authors suggest:

  • Interpolation: Using linear interpolation to estimate model values at specific observation coordinates significantly reduces error compared to simple grid-box averaging.
  • Aggregation: Aggregating multiple satellite pixels to fill a model grid box is effective but often limited by cloud cover.
  • Model Resolution: Global model resolution needs to reach ~50 km before spatial sampling errors become negligible.

Conclusion and Future Directions

This study assumes "perfect" temporal sampling. In reality, intermittent satellite passes and brief flight campaigns likely lead to even higher real-world errors. Future model evaluation should move toward "observation operators"—software that samples the model in the exact same spatial and temporal manner as the instrument—rather than comparing simple grid-box averages to point data.

Summary updated

Abstract

The spatial resolution of global climate models with interactive aerosol and the observations used to evaluate them is very different. Current models use grid spacings of ∼ 200 km, while satellite observations of aerosol use so-called pixels of ∼ 10 km. Ground site or airborne observations relate to even smaller spatial scales. We study the errors incurred due to different resolutions by aggregating high-resolution simulations (10 km grid spacing) over either the large areas of global model grid boxes ("perfect" model data) or small areas corresponding to the pixels of satellite measurements or the field of view of ground sites ("perfect" observations). Our analysis suggests that instantaneous root-mean-square (RMS) differences of perfect observations from perfect global models can easily amount to 30–160 %, for a range of observables like AOT (aerosol optical thickness), extinction, black carbon mass concentrations, PM2.5, number densities and CCN (cloud condensation nuclei). These differences, due entirely to different spatial sampling of models and observations, are often larger than measurement errors in real observations. Temporal averaging over a month of data reduces these differences more strongly for some observables (e.g. a threefold reduction for AOT), than for others (e.g. a twofold reduction for surface black carbon concentrations), but significant RMS differences remain (10–75 %). Note that this study ignores the issue of temporal sampling of real observations, which is likely to affect our present monthly error estimates. We examine several other strategies (e.g. spatial aggregation of observations, interpolation of model data) for reducing these differences and show their effectiveness. Finally, we examine consequences for the use of flight campaign data in global model evaluation and show that significant biases may be introduced depending on the flight strategy used.
BibTeX
@inproceedings{schutgens2016will,
  author = {Nick Schutgens and Edward Gryspeerdt and Natalie Weigum and Svetlana Tsyro and Daisuke Goto and Michael Schulz and Philip Stier},
  title = {{Will a perfect model agree with perfect observations? The impact of spatial sampling}},
  booktitle = {Atmospheric},
  year = {2016},
  doi = {10.5194/acp-16-6335-2016},
}
Figure