Affine-Invariant Local Descriptors and Neighborhood Statistics for Texture Recognition

Svetlana Lazebnik, Cordelia Schmid, Jean PonceICCV 2003 · OpenReview

Abstract

We present a framework for texture recognition based on local affine-invariant descriptors and their spatial layout. At modelling time, a generative model of local descriptors is learned from sample images using the EM algorithm. The EM framework allows the incorporation of unsegmented multitexture images into the training set. The second modelling step consists of gathering co-occurrence statistics of neighboring descriptors. At recognition time, initial probabilities computed from the generative model are refined using a relaxation step that incorporates co-occurrence statistics. Performance is evaluated on images of an indoor scene and pictures of wild animals.

Introduction

Texture recognition in real-world environments faces significant hurdles, primarily due to perspective distortion and scale variations. Traditional methods often rely on global statistics or fixed-size windows, which fail when the viewpoint changes or when an image contains multiple textures. This paper addresses two core challenges: achieving geometric invariance to handle warped appearances and enabling multi-texture segmentation to classify complex scenes where local information alone is ambiguous.

Methodology

The proposed framework utilizes a two-stage architecture for modeling and recognizing textures.

Feature Extraction and Shape Selection

To achieve affine invariance, the system moves away from fixed square windows. Instead, it employs an affine-adapted Laplacian blob detector. This detector automatically identifies "blobs" and determines their elliptical shapes. These ellipses are subsequently normalized into circles, effectively counteracting perspective warping before further analysis.

Local Descriptors

The normalized patches are represented using intensity domain spin images. These are 2D histograms capturing the distribution of brightness relative to the blob's center. This representation is inherently rotationally invariant and robust to fluctuations in lighting.

Generative Modeling via EM

The distribution of these local descriptors is modeled using a Gaussian Mixture Model (GMM). A key innovation is the use of the Expectation-Maximization (EM) algorithm for "weakly supervised" learning. This allows the model to be trained on images containing multiple textures without requiring precise, pixel-level manual labels.

Spatial Relaxation

Local descriptors can be noisy or ambiguous (e.g., a single white dot could belong to various textures). To resolve this, the system incorporates neighborhood statistics through relaxation labeling. This iterative process refines the initial probability of a blob's label based on the labels of its neighbors, enforcing a spatial smoothness constraint that favors coherent segments.

Experimental Evaluation

The framework was tested on two distinct datasets:

  1. Indoor Scene Dataset: Comprising seven texture classes (e.g., Brick, Carpet, Wood) photographed from diverse viewpoints. This set included 45 multi-texture test images to evaluate segmentation capabilities.
  2. Animals in the Wild: This dataset featured unsegmented images of cheetahs, giraffes, and zebras. It tested the model's ability to distinguish animal textures from cluttered, natural backgrounds.

Results and Discussion

The inclusion of spatial relaxation yielded dramatic improvements in classification accuracy. In the indoor dataset, the accuracy for "Brick" (T1) regions rose from 0.61 to 0.97, while "Floor 2" (T5) regions saw an increase from 0.45 to 0.95. The system successfully segmented perceptually similar textures that typically confound simpler models.

In the animal dataset, the model performed exceptionally well on zebras and cheetahs. However, it faced challenges with giraffes. This was attributed to the complex, non-blob-like nature of giraffe patterns and the presence of background elements like trees in the training data, which the single "background" class struggled to model effectively.

Conclusion

This research demonstrates that combining affine-invariant local descriptors with spatial context significantly enhances texture recognition and segmentation. While the system is robust, its performance remains sensitive to the quality of initial local classifications. Future work could involve more sophisticated background modeling—perhaps by partitioning the background into sub-classes like grass or sky—and exploring interest point detectors better suited for non-blob-like geometries.

Summary updated

Abstract

We present a framework for texture recognition based on local affine-invariant descriptors and their spatial layout. At modelling time, a generative model of local descriptors is learned from sample images using the EM algorithm. The EM framework allows the incorporation of unsegmented multitexture images into the training set. The second modelling step consists of gathering co-occurrence statistics of neighboring descriptors. At recognition time, initial probabilities computed from the generative model are refined using a relaxation step that incorporates co-occurrence statistics. Performance is evaluated on images of an indoor scene and pictures of wild animals.
BibTeX
@inproceedings{lazebnik2003affine,
  author = {Svetlana Lazebnik and Cordelia Schmid and Jean Ponce},
  title = {{Affine-Invariant Local Descriptors and Neighborhood Statistics for Texture Recognition}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2003},
  url = {https://openreview.net/forum?id=H1NSW-MObr},
}
Figure