Multiresolution Deep Belief Networks

Yichuan Tang, Abdel-rahman MohamedAISTATS 2012

Abstract

This paper introduces the Multiresolution Deep Belief Network (MrDBN), a hierarchical approach to unsupervised learning that integrates classical multiresolution analysis with Deep Belief Networks. By utilizing a Laplacian Pyramid to decompose signals into multiple frequency bands, the MrDBN processes data through parallel neural streams, significantly improving generative modeling and classification accuracy across computer vision and speech recognition tasks.

Introduction

Standard Deep Belief Networks (DBNs) typically process raw pixels at a single resolution. However, real-world objects exhibit structures at various scales—a forest appears as a uniform green mass from afar but reveals intricate leaf patterns up close. While multiresolution analysis is a cornerstone of classical computer vision, early deep learning models often expected networks to learn scale-invariance from scratch. The MrDBN addresses this by explicitly providing multiple resolutions as input, fostering better generalization and more robust generative representations.

The MrDBN Architecture

The core innovation is a multi-stream architecture that leverages Laplacian Pyramid decomposition. Instead of a single input, the signal is broken into high-frequency details ($f^0$), medium-frequency details ($g^0$), and a low-resolution base image ($v^2$).

The MrDBN architecture featuring parallel streams for different frequency components and top-level integration.

The network consists of three separate DBN streams, each specializing in one frequency component. For the first layer, the authors employ Gaussian-Binary Restricted Boltzmann Machines (GRBMs). A critical enhancement in this model is the ability to learn residual variances ($\Lambda$) of the visible units, rather than assuming fixed noise. This allows the model to distinguish between highly predictable regions (e.g., smooth skin) and detailed areas (e.g., eyes).

Visualization of the learned residual variances ($\Lambda$), showing higher precision requirements for detailed facial features.

Finally, a top-level RBM integrates the hidden activations from all streams. For supervised tasks, labels are concatenated at this stage to facilitate classification.

Experimental Evaluation

The MrDBN was evaluated across three distinct domains:

  • Toronto Face Database (TFD): Used to assess generative capabilities on over 100,000 grayscale face images.
  • NORB: A dataset of 3D toy objects under varying conditions, used for challenging classification.
  • TIMIT: A speech corpus where multiresolution was achieved via time-domain downsampling.

Key Results

The MrDBN demonstrated superior performance in both generative and discriminative tasks:

  • Generative Quality: The model produced significantly sharper and more realistic face samples than standard DBNs, capturing fine details like teeth and textures that baselines typically blur.
  • Classification: On the NORB dataset, the MrDBN achieved a 5.8% error rate, outperforming standard DBNs (8.3%) and Deep Boltzmann Machines (7.2%).
  • Speech Recognition: On the TIMIT core test set, it reached a Phone Error Rate (PER) of 20.3%, surpassing the then-state-of-the-art of 20.7%.

Conclusion

By forcing specialization across frequency bands, the MrDBN avoids the blurring effects common in standard generative models. Learning coarse structures from low-resolution inputs acts as a natural regularizer, preventing the model from being overwhelmed by high-frequency noise. While the augmented input space makes standard log-probability comparisons difficult, the improvements in reconstruction and classification are clear. Future work may explore visual search strategies that utilize high-resolution data only when necessary.

Summary updated

BibTeX
@article{tang2012multiresolution,
  author = {Yichuan Tang and Abdel-rahman Mohamed},
  title = {{Multiresolution Deep Belief Networks}},
  journal = {AISTATS},
  year = {2012},
}
Figure