Directional multiscale statistical modeling of images

D.d.-y. Po, N. MinhProceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE 2003 · DOI

TL;DR

This paper introduces a statistical model for the contourlet transform, a powerful image representation that excels at capturing directional features like edges. By analyzing the statistical dependencies of contourlet coefficients, the authors develop a Hidden Markov Tree (HMT) model that significantly improves image denoising and texture retrieval performance compared to wavelet-based methods.

What Problem Does This Paper Address?

Traditional image processing techniques often struggle to efficiently represent and analyze the smooth contours and edges that are fundamental to natural images. While wavelets offer multiresolution analysis, their separable nature limits their ability to capture directional information effectively. This leads to inefficient representations and suboptimal performance in tasks like denoising and texture analysis, especially for images with strong directional features. The paper aims to address this by developing a statistical model for the contourlet transform, a more recent, non-separable, and directionally rich image representation.

Methods

The core of this work lies in understanding and modeling the statistical properties of contourlet coefficients. The authors first conduct a detailed statistical analysis of these coefficients from natural images, examining their marginal and joint distributions using histograms and mutual information. They discover that contourlet coefficients are non-Gaussian marginally but become approximately Gaussian when conditioned on their neighbors.

Based on these findings, they propose a Contourlet Hidden Markov Tree (HMT) model. This model leverages the observed statistical dependencies:

  • Inter-scale dependencies: Similar to wavelet HMTs, it models relationships between coefficients at different resolutions.
  • Intra-subband dependencies: It captures correlations between coefficients within the same directional subband.
  • Inter-orientation dependencies: Crucially, it models dependencies between coefficients in different directional subbands at the same scale, a capability lacking in wavelet HMTs.

This comprehensive modeling allows the Contourlet HMT to capture the rich directional information inherent in the contourlet transform. The model is then applied to image denoising and texture retrieval tasks.

Datasets

The paper uses standard natural images for statistical analysis and evaluation. Specifically mentioned are:

  • "Peppers" image for detailed statistical analysis and visualization.
  • "Lena," "Barbara," and "Peppers" for mutual information estimation.
  • A set of 10 diverse images (Lena, Barbara, Peppers, Goldhill, Boat, Baboon, Couple, Zelda, Bridge, Truck) for denoising evaluation.
  • A texture database constructed from the Brodatz database, comprising 64 texture classes, with 512 texture images in total (128x128 sub-images).

No new benchmarks or datasets were introduced.

Key Insights and Ideas

  • Contourlets are directionally rich: Unlike wavelets, contourlets can represent smooth contours with significantly fewer coefficients due to their directional basis functions.
  • Conditional Gaussianity is key: While contourlet coefficients are non-Gaussian overall, they behave like Gaussian variables when conditioned on their local neighborhood (parents, neighbors, and cousins). This is a crucial insight for statistical modeling.
  • Inter-orientation dependencies matter: The ability of contourlets to capture information across different orientations is a significant advantage. The Contourlet HMT explicitly models these dependencies, which wavelet HMTs cannot.
  • Neighbors are most informative: Mutual information analysis reveals that coefficients are most dependent on their spatial neighbors within the same subband, followed by cousins (across orientations), and then parents (across scales).
  • Contourlet HMT outperforms Wavelet HMT: The explicit modeling of inter-orientation dependencies in the Contourlet HMT leads to superior performance in both denoising and texture retrieval, especially for images with strong directional features.

Key Results

  • Denoising: Contourlet HMT achieved higher Peak Signal-to-Noise Ratios (PSNR) than wavelet HMT and other classical methods (wiener2, wavelet thresholding) across multiple test images. For example, on the "Zelda" image, it achieved a PSNR of 27.707 dB, outperforming wavelet HMT (27.631 dB) and wavelet thresholding (26.046 dB). Visually, it produced smoother denoised images with better preservation of edges and oriented features.
  • Texture Retrieval: Contourlet HMT demonstrated an average retrieval rate approximately 2.5% higher than wavelet HMT. It particularly excelled at retrieving textures with diverse directional components, while wavelets performed better on textures with dominant vertical or horizontal features.

Limitations and Future Work

  • Computational Complexity: While not explicitly detailed as a limitation, the contourlet transform itself is generally more computationally intensive than the wavelet transform. The HMT model also adds to this complexity.
  • Filter Choice Sensitivity: The paper notes that mutual information estimates are sensitive to the choice of filters and directional partitioning schemes. This suggests that optimal performance might depend on careful selection of these parameters.
  • Future Work: The authors suggest that the Contourlet HMT model effectively captures directional information, which is valuable for image processing. This implies that further applications leveraging this directional capability could be explored. The paper also mentions the availability of software for implementing the model, facilitating further research.

Summary updated

Abstract

The contourlet transform is a new extension to the wavelet transform in two dimensions using non-separable and directional filter banks. The contourlet expansion is composed of basis images oriented at varying directions in multiple scales, with flexible aspect ratios. With this rich set of basis images, the contourlet transform can effectively capture the smooth contours that are the dominant features in natural images with only a small number of coefficients. We begin with a detail study of the statistics of the contourlet coefficients of natural images, using histogram estimates of the marginal and joint distributions, and mutual information measurements to characterize the dependencies between coefficients. The study reveals the non-Gaussian marginal statistics and strong intra-subband, cross-scale, and cross-orientation dependencies of contourlet coefficients. It is also found that conditioned on the magnitudes of their generalized neighborhood coefficients, contourlet coefficients can approximately be modeled as Gaussian variables with variances directly related to the generalized neighborhood magnitudes. Based on these statistics, we model contourlet coefficients using a hidden Markov tree (HMT) model that can capture all of their inter-scale, inter-orientation, and intra-subband dependencies. We experiment this model in the image denoising and texture retrieval applications where the results are very promising. In denoising, contourlet HMT outperforms wavelet HMT and other classical methods in terms of both peak signal-to-noise ratio (PSNR) and visual quality. In particular, it preserves edges and oriented features better than other existing methods. In texture retrieval, it shows improvements in performance over wavelet methods for various oriented textures.
BibTeX
@inproceedings{po2003directional,
  author = {D.D.-Y. Po and N. Minh},
  title = {{Directional multiscale statistical modeling of images}},
  booktitle = {Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE},
  year = {2003},
  doi = {10.1117/12.506412},
}
Figure