Pattern Field Classification with Style Normalized Transformation

Xu-Yao Zhang, Kaizhu Huang, Cheng-Lin LiuIJCAI 2011 · OpenReview

Abstract

Field classification is an extension of the traditional classification framework, by breaking the i.i.d. assumption. In field classification, patterns occur as groups (fields) of homogeneous styles. By utilizing style consistency, classifying groups of patterns is often more accurate than classifying single patterns. In this paper, we extend the Bayes decision theory, and develop the Field Bayesian Model (FBM) to deal with field classification. Specifically, we propose to learn a Style Normalized Transformation (SNT) for each field. Via the SNTs, the data of different fields are transformed to a uniform style space (i.i.d. space). The proposed model is a general and systematic framework, under which many probabilistic models can be easily extended for field classification. To transfer the model to unseen styles, we propose a transductive model called Transfer Bayesian Rule (TBR) based on self-training. We conducted extensive experiments on face, speech and a large-scale handwriting dataset, and got significant error rate reduction compared to the state-of-the-art methods.

Introduction

Most statistical pattern recognition models rely on the independent and identically distributed (i.i.d.) assumption, where every sample is treated as an independent event drawn from the same distribution. However, in many practical scenarios, patterns occur in homogeneous groups or "fields." For instance, a group of characters written by the same person shares a unique "writer style," and a sequence of words spoken by one person shares a specific accent. Within these fields, patterns are not independent because they share a style, and different fields are not identically distributed because styles vary between writers or speakers.

Comparison between (a) traditional singlet classification and (b) the proposed field classification framework.

Standard "singlet" classification ignores this rich group information. This research aims to bridge this gap by creating a framework that explicitly models and "normalizes" these styles to improve classification accuracy.

Methodology

The authors propose a general framework called the Field Bayesian Model (FBM), which centers on the separation of "style" from "content."

Style Normalized Transformation (SNT)

For every field, the model learns a specific affine transformation ($g_i(x) = A_i x + b_i$). This transformation acts as a "style-remover," mapping the data from a field-specific style into a universal, style-neutral "i.i.d. space."

Field Bayesian Model (FBM)

During the training phase, the model simultaneously learns the parameters for the underlying classifier (the "content") and the SNTs for all training fields. To prevent the model from distorting data excessively, a regularization term is employed to penalize large deviations from the identity matrix.

Transfer Bayesian Rule (TBR)

To handle unseen styles during testing, the authors introduced a transductive approach called the Transfer Bayesian Rule. When a new field arrives, the model uses an iterative self-training approach: it predicts labels, uses those labels to estimate the best SNT for that specific test field, and then refines the labels accordingly.

Experimental Evaluation

The FBM framework was evaluated across three distinct domains:

  1. Face Recognition: The Gourier et al. (2004) pose database, featuring 15 persons across 13 horizontal pose angles.
  2. Speech Recognition: A benchmark British English vowel dataset consisting of 11 vowels uttered by 15 speakers.
  3. Handwriting Recognition: The large-scale CASIA-OLHWDB dataset, involving 3,755 Chinese character classes and approximately 495,000 patterns from 100 different writers.

Results and Discussion

The FBM framework consistently outperformed traditional singlet classification and existing field models like the Bilinear Model or Style Mixture Model.

  • Face (Pose): Error rates dropped from 40.00% (standard NCM) to 21.33% using Field-NCM with TBR.
  • Speech (Vowels): The model achieved a 21.65% error rate, outperforming the previously reported best of 22.70% for the bilinear model.
  • Handwriting: On the massive 3,755-class Chinese character set, the standard MQDF classifier had a 12.17% error rate. The Field-QDF model reduced this to 8.89%.
Error rate comparison across different writers for various model configurations.

The results demonstrate that treating style as a "context" rather than noise provides valuable information about the dependencies between patterns in a group. The TBR approach is particularly effective as it allows the model to adapt to a completely new writer or speaker on the fly.

Conclusion

This paper introduces a systematic framework for field classification by learning style-normalized transformations. By mapping diverse styles into a uniform space, the Field Bayesian Model significantly boosts classification accuracy. Future work will explore extending this framework to non-probabilistic classifiers, such as Support Vector Machines (SVMs), and investigating more complex, non-Gaussian class-conditional distributions.

Summary updated

Abstract

Field classification is an extension of the traditional classification framework, by breaking the i.i.d. assumption. In field classification, patterns occur as groups (fields) of homogeneous styles. By utilizing style consistency, classifying groups of patterns is often more accurate than classifying single patterns. In this paper, we extend the Bayes decision theory, and develop the Field Bayesian Model (FBM) to deal with field classification. Specifically, we propose to learn a Style Normalized Transformation (SNT) for each field. Via the SNTs, the data of different fields are transformed to a uniform style space (i.i.d. space). The proposed model is a general and systematic framework, under which many probabilistic models can be easily extended for field classification. To transfer the model to unseen styles, we propose a transductive model called Transfer Bayesian Rule (TBR) based on self-training. We conducted extensive experiments on face, speech and a large-scale handwriting dataset, and got significant error rate reduction compared to the state-of-the-art methods.
BibTeX
@inproceedings{zhang2011pattern,
  author = {Xu-Yao Zhang and Kaizhu Huang and Cheng-Lin Liu},
  title = {{Pattern Field Classification with Style Normalized Transformation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2011},
  url = {https://openreview.net/forum?id=rkV-hXGO-B},
}
Figure