Abstract
This paper presents a probabilistic part-based approach for texture and object recognition. Textures are represented using a part dictionary found by quantizing the appearance of scale- or affine-invariant keypoints. Object classes are represented using a dictionary of composite semi-local parts, or groups of neighboring keypoints with stable and distinctive appearance and geometric layout. A discriminative maximum entropy framework is used to learn the posterior distribution of the class label given the occurrences of parts from the dictionary in the training set. Experiments on two texture and two object databases demonstrate the effectiveness of this framework for visual classification.
Introduction
In the mid-2000s, computer vision research began shifting toward "bag-of-features" models, which represent images as unordered sets of local descriptors. While effective, these models often ignored the spatial relationships between features. Conversely, generative models that accounted for structure, such as Markov Random Fields, were often too computationally complex for large-scale inference. This work introduces a discriminative maximum entropy (MaxEnt) framework that serves as a middle ground, handling multiple feature types and their inter-relations without the restrictive independence assumptions of Naive Bayes classifiers.
Methodology
The core of the proposed framework is the modeling of the posterior distribution $P(c|I)$—the probability of a class $c$ given an image $I$. The goal is to find a distribution that matches the statistics of the training data while remaining as unbiased as possible.
Representation and Learning
The framework results in an exponential form where class-specific weights are learned for different feature functions. For texture representation, scale- and affine-invariant keypoints are used to build a "texton dictionary" via clustering. Feature functions are defined based on texton frequencies and co-occurrence (bigrams) of adjacent textons.
For object representation, the authors utilize semi-local parts—geometrically stable configurations of multiple keypoints that capture local rigid structure. Parameters are optimized using the Improved Iterative Scaling (IIS) algorithm, with Laplace smoothing applied to handle data sparsity.
Experimental Evaluation
The framework was evaluated on four distinct datasets: the Brodatz and UIUC texture databases, a CalTech object subset, and a custom Birds dataset featuring six species.
Results
The MaxEnt model consistently outperformed baseline Naive Bayes and nearest-neighbor classifiers. On the UIUC database, the model achieved 97.41% accuracy, significantly exceeding the previous state-of-the-art of 92.61%. On the Brodatz database, it reached 87.37%. Object recognition performance was also high, with 96.38% on the CalTech subset and 92.33% on the Birds dataset.
Discussion and Insights
A key finding was the robustness of the "bag-of-words" approach; adding spatial co-occurrence relations did not significantly improve classification over simple part frequencies. This suggests that for many categories, the presence of specific parts is a stronger cue than their exact relative placement. Furthermore, by focusing on $P(c|I)$ rather than the joint distribution $P(I, c)$, the model avoids the intractability issues common in generative vision models.
Limitations and Future Work
Despite its success, the framework has limitations. The current MATLAB implementation for computing semi-local parts is computationally expensive, requiring significant processing time for large databases. Additionally, while spatial relations showed limited impact on classification accuracy, they remain crucial for localization tasks. Future work may involve more sophisticated feature selection methods to further enhance the dictionary representation.
Summary updated