Abstract
Prior distributions of binarized natural images are learned by using a Boltzmann machine. According to the results of this study, there emerges a structure with two sublattices in the interactions, and the nearest-neighbor and next-nearest-neighbor interactions correspondingly take two discriminative values, which reflects the individual characteristics of the three sets of pictures that we process. Meanwhile, in a longer spatial scale, a longer-range, although still rapidly decaying, ferromagnetic interaction commonly appears in all cases. The characteristic length scale of the interactions is universally up to approximately four lattice spacings $\xi \approx 4$. These results are derived by using the mean-field method, which effectively reduces the computational time required in a Boltzmann machine. An improved mean-field method called the Bethe approximation also gives the same results, as well as the Monte Carlo method does for small size images. These reinforce the validity of our analysis and findings. Relations to criticality, frustration, and simple-cell receptive fields are also discussed.
Introduction
In Bayesian image processing, algorithms rely heavily on "prior distributions"—mathematical models that define the expected characteristics of a "natural" image. Historically, researchers have often used a single, generic prior for all image types. This paper investigates whether specific categories of images (such as faces versus forests) have distinct prior distributions that could be leveraged for improved processing. By using a fully connected Boltzmann machine (an Ising model without hidden layers), the study aims to learn and interpret the pairwise pixel correlations of different image categories to identify unique visual characteristics.
Methodology
Model and Learning Task
The authors utilize a fully connected Boltzmann machine to perform maximum likelihood estimation via moment matching. The goal is to match the model's mean pixel values (magnetizations) and pairwise correlations to empirical data derived from real images.
Datasets and Preprocessing
The empirical distributions were generated by cutting binarized images into square patches of varying sizes ($L=8, 16, 32$). Three distinct datasets were used:
- Human face pictures (from the Color FERET Database).
- Aerial geographic pictures (from the Geospatial Information Authority of Japan).
- Forest pictures (from the SUN Database).
All images were binarized into $+1/-1$ spins using spatial color quantization and Riemersma dithering.
Computational Approximations
Because calculating the exact probability distribution is computationally intractable for large patches, the authors employed two mean-field approximations: the Naive Mean-Field (NMF) method and the Bethe Approximation (BA). These were validated against computationally expensive Monte Carlo (MC) simulations for small $8 \times 8$ patches, showing nearly identical results.
Results and Discussion
Sublattice Structures and Universal Decay
A key finding is the emergence of a "checkerboard" sublattice structure in short-range interactions. The strengths and signs (ferromagnetic vs. antiferromagnetic) of nearest and next-nearest neighbor links are highly specific to the image category. For example, faces show negative nearest-neighbor interactions due to high-contrast edges, while aerial shots are entirely positive. These structures are heavily influenced by the dithering process used during binarization.
Regardless of the category, interactions beyond the immediate neighbors are universally positive and decay exponentially. The characteristic length scale is surprisingly short, vanishing beyond a distance of approximately 4 pixels ($\xi \approx 4$).
The Missing Criticality
Natural images are known for exhibiting scale-free "criticality." Surprisingly, the learned Boltzmann machines do not exhibit this property. This implies that criticality in natural images must stem from higher-order statistics, which a standard pairwise Boltzmann machine cannot capture.
Conclusion
The study proposes a simplified 6-parameter prior model for natural images: four parameters for alternating short-range interactions and two for the exponential decay of longer-range interactions. While these category-specific priors offer a more nuanced model than generic alternatives, the failure to capture criticality highlights the necessity of developing functional forms that incorporate higher-order statistics in future generative image models.
Summary updated