Towards a Closed Form Second-Order Natural Scene Statistics Model

Zeina Sinno, Constantine Caramanis, Alan C. BovikIEEE TIP 2018 · OpenReview

Abstract

Previous work on natural scene statistics (NSS)-based image models has focused primarily on characterizing the univariate bandpass statistics of single pixels. These models have proven to be powerful tools driving a variety of computer vision and image/video processing applications, including depth estimation, image quality assessment, and image denoising, among others. Multivariate NSS models descriptive of the joint distributions of spatially separated bandpass image samples have, however, received relatively little attention. Here, we develop a closed form bivariate spatial correlation model of bandpass and normalized image samples that completes an existing 2D joint generalized Gaussian distribution model of adjacent bandpass pixels. Our model is built using a set of diverse, high-quality naturalistic photographs, and as a control, we study the model properties on white noise. We also study the way the model fits are affected when the images are modified by common distortions.

Introduction

Natural Scene Statistics (NSS) are the statistical regularities inherent in images of the natural world. Human visual systems are highly optimized to process these regularities. In computer vision, univariate NSS models—which describe the behavior of individual pixels after filtering—have been widely successful in tasks like denoising. However, these models often ignore the bivariate relationship: how a pixel relates to its neighbors across various distances and angles.

Existing attempts to model these spatial relationships have often lacked a "closed-form" representation—a precise mathematical equation that can be systematically applied. Without such a model, it is difficult to analyze how specific distortions, such as motion blur or compression, degrade the "naturalness" of an image. This research addresses this gap by providing a parametric model for the spatial correlation of bandpass-filtered, normalized image samples.

Methodology

The proposed approach utilizes a multi-step pipeline to extract and model second-order statistics from natural images.

Figure 1: Processing pipeline for extracting second-order natural scene statistics.
  1. Steerable Filtering: Images are processed through filters responding to specific orientations and scales, mimicking the "simple cells" in the primary visual cortex.
  2. Divisive Normalization: A non-linear step that "Gaussianizes" the data by dividing pixel responses by the energy of their neighbors, facilitating mathematical modeling.
  3. Bivariate Modeling: A Multivariate Generalized Gaussian Distribution (MGGD) is employed to describe the joint behavior of pixel pairs.
  4. Correlation Model: The core contribution is a model for the Pearson correlation ($\rho$) between pixels separated by distance $d$. The study found that this correlation follows a stabilized reciprocal power law.

The model was validated using several datasets, including the LIVE Image Quality Assessment Database, the CSIQ Database, and the Toyama Database. Simulated white noise was used as a control to distinguish natural structural signatures from filter artifacts.

Key Insights and Results

The research reveals that spatial correlation in natural scenes decays according to a reciprocal power law ($1/|d|^\beta$), indicating a fundamental structural regularity. Furthermore, these statistics exhibit scale invariance, remaining consistent across different viewing scales.

Figure 4: Correlation $\rho$ as a function of orientation difference $\theta_2 - \theta_1$ across various spatial distances $d$.

The closed-form model achieves high fidelity, fitting empirical data with a Mean Squared Error on the order of $10^{-3}$. Crucially, the model parameters are sensitive to image distortions. When used as features for a Support Vector Machine (SVM), the model achieved 70-71% accuracy in identifying distortion types (e.g., blur, JPEG compression, noise). This demonstrates that distortions alter the second-order statistics in predictable, systematic ways.

Conclusion

While the model faces limitations at very small scales ($\sigma=1$) due to filter degeneracy and high-frequency noise, it provides a robust framework for understanding image structure. Future work may extend this luminance-based model to chromatic statistics and apply it to blind image quality assessment and depth estimation, where the systematic change in correlations can infer 3D scene structure.

Summary updated

Abstract

Previous work on natural scene statistics (NSS)-based image models has focused primarily on characterizing the univariate bandpass statistics of single pixels. These models have proven to be powerful tools driving a variety of computer vision and image/video processing applications, including depth estimation, image quality assessment, and image denoising, among others. Multivariate NSS models descriptive of the joint distributions of spatially separated bandpass image samples have, however, received relatively little attention. Here, we develop a closed form bivariate spatial correlation model of bandpass and normalized image samples that completes an existing 2D joint generalized Gaussian distribution model of adjacent bandpass pixels. Our model is built using a set of diverse, high-quality naturalistic photographs, and as a control, we study the model properties on white noise. We also study the way the model fits are affected when the images are modified by common distortions.
BibTeX
@article{sinno2018towards,
  author = {Zeina Sinno and Constantine Caramanis and Alan C. Bovik},
  title = {{Towards a Closed Form Second-Order Natural Scene Statistics Model}},
  journal = {IEEE Trans. Image Process.},
  year = {2018},
  url = {https://openreview.net/forum?id=c74XvENVu4Y},
}
Figure