Estimating illumination direction of three-dimensional surface texture based on active basis and Mojette transform

Junyu Dong, Liyuan Su, Yu Zhang, Florent Autrusseau, Zhanbin Yang

TL;DR

This paper introduces a novel method for estimating the direction of light illuminating 3D surfaces, even for complex, anisotropic textures. By combining the Mojette transform for texture orientation analysis with an Active Basis model for feature extraction, the approach can accurately determine illumination direction without requiring the specific texture to be present in the training data.

What Problem Does This Paper Address?

Estimating the direction of illumination on a 3D surface from a single image is a challenging problem, especially for anisotropic textures where the surface's own directionality interacts with the lighting. Previous methods often assumed isotropic surfaces or required the exact texture to be in the training set, limiting their applicability. This paper aims to overcome these limitations by developing a robust method that can handle anisotropic textures and generalize to unseen textures with similar orientations.

Methods

The proposed method employs a two-stage approach:

  1. Texture Orientation Estimation: The Mojette transform is used to analyze the directional trends within a surface texture. By computing projections at various angles and analyzing their variance, the dominant texture orientations can be identified. This is achieved by finding the angles where the variance of the Mojette projections is maximized. The second derivative of the variance vector is used to pinpoint the principal texture direction.
  2. Illumination Direction Estimation:
    • Active Basis Model: An Active Basis model, composed of Gabor wavelet elements (or Haar features for efficiency), is learned from training images. This model captures the "edge" information and implicit illumination cues within the texture.
  3. Feature Comparison: For a given test image, its texture orientation is first determined using the Mojette transform. Then, a set of training textures with similar orientations are identified by comparing their Mojette transform vectors.
  4. Histogram Matching: Base histograms are generated from the Active Basis models of the test and similar training textures. The illumination direction is estimated by finding the training sample whose base histogram has the highest correlation coefficient with the test image's histogram.

Datasets

The primary dataset used for evaluation is the PhoTex texture database. This database contains various 3D surface textures, each represented by multiple images captured under different illumination conditions (varying zenithal and azimuthal angles). The authors also mention the Brodaz database but did not use it due to the lack of known illumination information.

Key Insights and Ideas

  • Mojette Transform for Texture Orientation: The Mojette transform proves to be an effective tool for robustly identifying dominant texture orientations, even in complex anisotropic surfaces.
  • Active Basis for Illumination Cues: The Active Basis model, by capturing edge information and its response to light, effectively encodes cues about the illumination direction.
  • Generalization to Unseen Textures: A key innovation is the ability to estimate illumination direction for textures not explicitly present in the training set, as long as they share similar orientation features. This is achieved by first matching texture orientations and then comparing feature histograms.
  • Correlation Coefficient Superiority: The study found that using the correlation coefficient for comparing base histograms yields better illumination direction estimation accuracy compared to Euclidean distance.

Key Results

  • Texture Orientation Estimation: The proposed Mojette transform-based method outperforms traditional Radon transform-based methods for texture orientation estimation, as shown in Figure 9.
  • Illumination Direction Estimation Accuracy:
    • When the test texture class is included in the training set, the method achieves an accuracy of 80.3% with the best parameter combination.
  • The correlation coefficient metric consistently outperforms Euclidean distance for similarity measurement.
  • The method demonstrates effectiveness even when the test texture class is not included in the training set, a significant advantage over prior work.
  • Parameter Sensitivity: The accuracy of illumination direction estimation is influenced by parameters like the number of orientations (N), Haar feature size, and the number of bases. Optimal parameters were identified (N=12, haarSize=13x13, baseNumber=300).
  • Limitations: Certain anisotropic surfaces, particularly when illumination aligns with surface orientation, can lead to larger estimation errors due to weak texture response.

Limitations and Future Work

  • Anisotropic Surface Challenges: For some anisotropic surfaces where illumination aligns with surface orientation, estimation errors can be larger. This is attributed to the weak response of the texture structure to certain illumination directions.
  • Mojette Transform Variability: The angular energy distribution within a single texture can change significantly, making it not always reliable to find the dominant orientation using all frequencies.
  • Future Work: The authors suggest further investigation into other features based on active basis and relevant similarity measurements to improve estimation accuracy. Exploring how to better handle situations where illumination aligns with surface orientation is also a promising direction.

Summary updated

Abstract

Given an image of a rough surface texture, the approach described here can estimate the direction from which the surface was lit. Unlike previous work, we require neither surface isotropy nor that the texture is included in the training set. The approach is based on active basis and the Mojette transform. The Mojette transform is used to estimate the orientation of the texture; it does so by finding which training samples have similar orientation features. The active basis model is then learned from training images by the shared pursuit algorithm. Next, the base histograms of the test image and textures with similar orientation features in the training set are compared so that the illumination directions can be estimated by minimizing their correlation coefficients.
BibTeX
@article{dong2012estimating,
  author = {Junyu Dong and Liyuan Su and Yu Zhang and Florent Autrusseau and Zhanbin Yang},
  title = {{Estimating illumination direction of three-dimensional surface texture based on active basis and Mojette transform}},
  journal = {J. Electronic Imaging},
  year = {2012},
  doi = {10.1117/1.jei.21.1.013023},
}
Figure