Abstract
Standard machine learning models often struggle with high-dimensional, noisy data like handwritten characters, especially when distinguishing between highly similar classes. This paper presents Explanation-Based Feature Construction (EBFC), a method that leverages domain-specific knowledge—specifically stroke models for Chinese characters—to guide the creation of discriminative features. By identifying stable "reference strokes" to establish a spatial frame, the system can isolate and focus on the subtle differences that distinguish similar characters. Experimental results on the ETL9B dataset demonstrate that EBFC significantly outperforms standard Support Vector Machines (SVMs) on the most challenging character pairs.
Introduction
In supervised learning, the quality of features often determines the success of the model. For tasks like handwritten character recognition, raw pixel data is often too noisy and high-dimensional for efficient learning. While standard approaches like SVMs try to find complex non-linear combinations of these pixels, they often require massive training sets to avoid overfitting. This research explores how "explanations"—mappings of raw data to high-level domain theories—can automate the construction of robust, high-level features that bridge the gap between raw pixels and structural concepts like strokes.
Methodology: Explanation-Based Feature Construction (EBFC)
The core of EBFC is the integration of a "domain theory" (e.g., a model of how characters are composed of strokes) into the feature extraction process. The method proceeds through four stages: explanation (mapping pixels to stroke models), identification (distinguishing reference vs. information-bearing strokes), statistical evaluation of stroke reliability, and optimization of a "target window."
By finding reliable reference strokes first, the system creates a stable frame of reference. This allows it to precisely calculate where the most informative (discriminative) part of the character should be, even if the handwriting is messy or shifted. This "knowledge-guided focus" ensures the model learns features for the correct structural reasons rather than coincidental pixel patterns.
Experimental Results
The researchers evaluated EBFC using the ETL9B database, focusing on the 100 most difficult character pairs. They compared a standard SVM using Weighted Direction Code Histograms (WDH) against their enhanced SVM(EBL) approach.
As shown in the results table, the EBL-guided feature construction led to dramatic improvements in accuracy for the most confusing pairs. For instance, in Pair 14, the error rate dropped from 7.0% to 2.0%. In Pair 4, a particularly difficult case, the error rate was more than halved, falling from 13.0% to 5.8%. The EBFC approach effectively "cleans" the input by focusing the classifier's attention only on the most informative regions.
Conclusion
EBFC demonstrates that incorporating domain knowledge into the feature construction process can significantly enhance the performance of statistical classifiers. By using similarities to find differences, the system creates a stable frame of reference that allows it to "zoom in" on the specific regions where characters differ. While the method currently requires a pre-existing domain theory and has higher computational costs during training, its ability to generalize from fewer examples on complex tasks marks a significant step toward more structural feature engineering.
Summary updated