PCANet: A Simple Deep Learning Baseline for Image Classification?

Tsung-Han Chan, Kui Jia, Shenghua Gao, Jiwen Lu, Zinan Zeng, Yi Ma

Abstract

In this work, we propose a very simple deep learning network for image classification which comprises only the very basic data processing components: cascaded principal component analysis (PCA), binary hashing, and block-wise histograms. In the proposed architecture, PCA is employed to learn multistage filter banks. It is followed by simple binary hashing and block histograms for indexing and pooling. This architecture is thus named as a PCA network (PCANet) and can be designed and learned extremely easily and efficiently. For comparison and better understanding, we also introduce and study two simple variations to the PCANet, namely the RandNet and LDANet. They share the same topology of PCANet but their cascaded filters are either selected randomly or learned from LDA. We have tested these basic networks extensively on many benchmark visual datasets for different tasks, such as LFW for face verification, MultiPIE, Extended Yale B, AR, FERET datasets for face recognition, as well as MNIST for hand-written digits recognition. Surprisingly, for all tasks, such a seemingly naive PCANet model is on par with the state of the art features, either prefixed, highly hand-crafted or carefully learned (by DNNs). Even more surprisingly, it sets new records for many classification tasks in Extended Yale B, AR, FERET datasets, and MNIST variations. Additional experiments on other public datasets also demonstrate the potential of the PCANet serving as a simple but highly competitive baseline for texture classification and object recognition.

Introduction

Deep Convolutional Neural Networks (ConvNets) have achieved remarkable success in image classification, yet they are often criticized as "black boxes" requiring massive datasets, extensive hyperparameter tuning, and significant computational resources for training via Stochastic Gradient Descent (SGD). This research investigates a fundamental question: how much of this complexity is truly necessary for high-level performance? By stripping away complex backpropagation and non-linear activations, the authors propose PCANet—a "naive" model designed to serve as a simple but highly competitive baseline for the computer vision community.

Methods

PCANet simplifies the deep learning pipeline into three primary stages:

  1. Cascaded PCA: Instead of iterative learning via backpropagation, the network extracts patches from input images and applies Principal Component Analysis (PCA) to identify lead eigenvectors, which serve as convolutional filters. This process is cascaded into a second stage to capture higher-order features.
  2. Binary Hashing: The output of the second stage undergoes binarization via a simple Heaviside step function. This non-linear mapping converts continuous filter responses into binary strings.
  3. Block-wise Histograms: Binary strings are converted to decimal values. The image is then partitioned into blocks, and local histograms of these values are computed and concatenated. This stage provides essential translation invariance and forms the final feature vector for a linear classifier, such as a Support Vector Machine (SVM).
The PCANet architecture showing cascaded PCA filters, binarization, and histogram generation.

To further understand the architecture's properties, the authors also introduced RandNet (using random filters) and LDANet (using filters derived from Linear Discriminant Analysis).

Results and Performance

The PCANet architecture was evaluated across diverse benchmarks, including face recognition (Extended Yale B, AR, FERET), digit recognition (MNIST), and object classification (CIFAR-10).

Key results include:

  • Extended Yale B: Achieved a near-perfect 99.58% accuracy, demonstrating extreme robustness to lighting variations.
  • FERET: Set a state-of-the-art average accuracy of 97.25%.
  • MNIST: Produced a competitive 0.62% error rate, matching far more complex models.
  • Efficiency: Training on 100,000 images took only 30 minutes, compared to 6 hours for a standard ConvNet, thanks to the closed-form nature of PCA.
Comparison of recognition rates for PCANet, RandNet, and LDANet as the number of filters increases.

As shown in the comparison above, PCANet consistently outperforms RandNet and often matches or exceeds LDANet as the number of filters in the first stage increases, highlighting the effectiveness of PCA-based filter learning.

Discussion and Conclusion

PCANet demonstrates that high performance in image classification can be achieved without traditional deep learning complexities like backpropagation or intermediate non-linearities. By deferring non-linearity and pooling to the final hashing and histogram stages, the model remains computationally efficient and robust to occlusions and noise.

However, limitations remain. The feature vector size grows exponentially with depth, making it difficult to scale to the extreme depths of modern architectures. Additionally, while it excels on aligned datasets, it faces challenges with the extreme scale and pose variations found in unconstrained datasets like ImageNet. Future work may involve exploring tensor-based filters or improved pre-processing to enhance generalization to more complex visual tasks.

Summary updated

Abstract

In this work, we propose a very simple deep learning network for image classification which comprises only the very basic data processing components: cascaded principal component analysis (PCA), binary hashing, and block-wise histograms. In the proposed architecture, PCA is employed to learn multistage filter banks. It is followed by simple binary hashing and block histograms for indexing and pooling. This architecture is thus named as a PCA network (PCANet) and can be designed and learned extremely easily and efficiently. For comparison and better understanding, we also introduce and study two simple variations to the PCANet, namely the RandNet and LDANet. They share the same topology of PCANet but their cascaded filters are either selected randomly or learned from LDA. We have tested these basic networks extensively on many benchmark visual datasets for different tasks, such as LFW for face verification, MultiPIE, Extended Yale B, AR, FERET datasets for face recognition, as well as MNIST for hand-written digits recognition. Surprisingly, for all tasks, such a seemingly naive PCANet model is on par with the state of the art features, either prefixed, highly hand-crafted or carefully learned (by DNNs). Even more surprisingly, it sets new records for many classification tasks in Extended Yale B, AR, FERET datasets, and MNIST variations. Additional experiments on other public datasets also demonstrate the potential of the PCANet serving as a simple but highly competitive baseline for texture classification and object recognition.
BibTeX
@article{chan2015pcanet,
  author = {Tsung-Han Chan and Kui Jia and Shenghua Gao and Jiwen Lu and Zinan Zeng and Yi Ma},
  title = {{PCANet: A Simple Deep Learning Baseline for Image Classification?}},
  journal = {IEEE Trans. Image Process.},
  year = {2015},
  url = {https://openreview.net/forum?id=r3VO6Bpnul9},
}

Loading related content...

Figure