Abstract
We classify digits of real-world house numbers using convolutional neural networks (ConvNets). Unlike many popular vision approaches that are hand-designed, ConvNets can automatically learn a unique set of features optimized for a given task. We augmented the traditional ConvNet architecture by learning multi-stage features and by using Lp pooling, establishing a new state-of-the-art of 94.85% accuracy on the SVHN dataset (45.2% error improvement).
Introduction
In 2012, the transition from hand-crafted computer vision features to fully learned deep learning systems marked a pivotal shift in the field. This research from the NYU Courant Institute demonstrated that Convolutional Neural Networks (ConvNets) could master the challenges of natural scene digit recognition. By applying specific architectural enhancements to the Street View House Numbers (SVHN) dataset, the authors proved that learned features could significantly outperform traditional methods like HOG or template matching.
The SVHN Dataset
The Street View House Numbers (SVHN) dataset represents a significant step up in difficulty from the classic MNIST benchmark. It contains over 600,000 labeled digits extracted from Google Street View images. These 32x32 pixel cropped color images present real-world challenges including low resolution, motion blur, varying illumination, and cluttered backgrounds.
Methodology
The authors utilized a 2-stage ConvNet architecture with two primary innovations: Multi-Stage Features and Lp Pooling.
Multi-Stage Features (MS)
In a standard ConvNet, only the final layer's output is typically fed into the classifier. This paper introduces a "multi-stage" approach where the output of the first stage is branched out, subsampled, and concatenated with the second stage's output. This ensures the classifier receives both high-level abstract shapes and fine-grained local textures.
Generalized Lp Pooling
Standard networks typically use Max or Average pooling. This work utilizes Lp pooling, defined as $O = (\sum I^p \times G)^{1/p}$. This generalization allows the network to tune the pooling behavior:
- $p=1$ approximates Gaussian averaging.
- $p=\infty$ approximates Max-pooling.
By using intermediate values, the network balances the preservation of strong features with noise suppression.
Results and Analysis
The proposed model achieved a state-of-the-art accuracy of 94.85%, representing a 45.2% improvement in error rate over previous non-ConvNet approaches. This brought machine performance significantly closer to the estimated human accuracy of 98% in unconstrained environments.
A key insight was the performance of "in-between" pooling. As shown in the validation error analysis, neither $p=1$ (average) nor very high $p$ (max) was optimal. Instead, values between 2 and 12 provided the best results, suggesting that a weighted focus on strong signals is superior to simple averaging or extreme max-pooling.
Conclusion
This work highlights the power of end-to-end feature learning. While multi-stage features provided varying gains depending on the task—offering more significant improvements for complex textures like pedestrian detection than for house numbers—the combination of architectural flexibility and large-scale data proved decisive. Future work suggests that artificial scale deformations and unsupervised pre-training could further bridge the gap to human-level perception.
Summary updated