A Nonparametric Bayesian Approach toward Stacked Convolutional Independent Component Analysis

Sotirios P. Chatzis, Dimitrios I. Kosmopoulos

Abstract

Unsupervised feature learning algorithms based on convolutional formulations of independent components analysis (ICA) have been demonstrated to yield state-of-the-art results in several action recognition benchmarks. However, existing approaches do not allow for the number of latent components (features) to be automatically inferred from the data in an unsupervised manner. This is a significant disadvantage of the state-of-the-art, as it results in considerable burden imposed on researchers and practitioners, who must resort to tedious cross-validation procedures to obtain the optimal number of latent features. To resolve these issues, in this paper we introduce a convolutional nonparametric Bayesian sparse ICA architecture for overcomplete feature learning from high-dimensional data. Our method utilizes an Indian buffet process prior to facilitate inference of the appropriate number of latent features under a hybrid variational inference algorithm, scalable to massive datasets. As we show, our model can be naturally used to obtain deep unsupervised hierarchical feature extractors, by greedily stacking successive model layers, similar to existing approaches. In addition, inference for this model is completely heuristics-free, thus, it obviates the need of tedious parameter tuning, which is a major challenge most deep learning approaches are faced with. We evaluate our method on several action recognition benchmarks, and exhibit its advantages over the state-of-the-art.

Introduction

In unsupervised feature learning, determining the optimal number of latent components—the "features"—is a persistent challenge. Traditional methods like Sparse Coding or Independent Component Analysis (ICA) typically require researchers to pre-define this number. Selecting too few features leads to under-representation, while too many can cause redundancy and computational inefficiency. Finding the "Goldilocks" number usually involves exhaustive cross-validation, which is both time-consuming and resource-intensive.

This paper introduces a "self-sizing" deep learning architecture designed to address this bottleneck. By integrating ICA with a nonparametric Bayesian prior, specifically the Indian Buffet Process (IBP), the proposed model automatically determines the appropriate number of latent features directly from the data. This approach eliminates the need for manual tuning and provides a heuristics-free framework for action recognition.

Methodology: SC-IBP-ICA

The core of the proposed method is SC-IBP-ICA (Stacked Convolutional Indian Buffet Process Independent Component Analysis). The architecture is built upon several key components:

  1. Independent Component Analysis (ICA): Serves as the primary engine for identifying statistically independent patterns, such as edges or motion gradients, within the data.
  2. Indian Buffet Process (IBP): Acts as a nonparametric Bayesian prior. Analogous to a buffet where customers (data points) select from an infinite array of dishes (features), the IBP allows the model to theoretically support an infinite number of features while only activating a finite, data-driven subset.
  3. Stick-Breaking Construction and Variational Inference: To ensure scalability, the authors employ a stick-breaking construction of the IBP. This allows for the use of Variational Inference, a fast approximation technique that makes the model suitable for massive datasets, avoiding the slow convergence of traditional MCMC methods.
  4. Convolutional Stacking: The model processes small spatiotemporal patches from video data. These layers are stacked greedily, enabling the network to learn a hierarchy of features—from simple motion edges in the initial layer to complex action representations in subsequent layers.
The SC-IBP-ICA architecture illustrating the hierarchical stacking of layers and pooling operations over spatiotemporal video patches.

Experimental Evaluation

The SC-IBP-ICA model was benchmarked against three standard action recognition datasets: Hollywood2 (movie clips), YouTube (diverse action categories), and KTH (human actions like walking and jogging).

Performance and Automatic Complexity Control

A key advantage of the SC-IBP-ICA model is its ability to "discover" the necessary model complexity. For instance, on the YouTube dataset, the model automatically converged to approximately 291 features for the first layer and 197 for the second. Similar behavior was observed on the KTH dataset, where it achieved 93.4% accuracy with 293 and 195 features in the first and second layers, respectively.

Performance comparison on the KTH dataset. The numbers in parentheses for SC-IBP-ICA indicate the automatically inferred number of features for each layer.

The model achieved results comparable to state-of-the-art unsupervised methods:

  • Hollywood2: 53.5% mean average precision (mAP).
  • YouTube: 75.4% accuracy.
  • KTH: 93.4% accuracy.

Notably, these results were achieved without the exhaustive parameter searches required by competing methods like Independent Subspace Analysis (ISA). Furthermore, the model demonstrated strong efficiency, with inference times ranging from 0.12 to 0.38 seconds per frame.

Discussion and Conclusion

The SC-IBP-ICA framework represents a significant step toward truly unsupervised, heuristic-free deep learning. By leveraging Bayesian nonparametrics, the model handles overcomplete representations naturally and generalizes well across different datasets. For example, features learned on KTH were found to transfer effectively to the Hollywood2 dataset, outperforming standard ISA in transfer learning scenarios.

While the current implementation relies on greedy layer-wise training and focuses on small video patches, it provides a robust foundation for future work in end-to-end Bayesian deep learning and high-resolution video processing. The primary contribution remains the elimination of the "number of features" problem, significantly reducing the human effort required to deploy high-performance action recognition systems.

Summary updated

Abstract

Unsupervised feature learning algorithms based on convolutional formulations of independent components analysis (ICA) have been demonstrated to yield state-of-the-art results in several action recognition benchmarks. However, existing approaches do not allow for the number of latent components (features) to be automatically inferred from the data in an unsupervised manner. This is a significant disadvantage of the state-of-the-art, as it results in considerable burden imposed on researchers and practitioners, who must resort to tedious cross-validation procedures to obtain the optimal number of latent features. To resolve these issues, in this paper we introduce a convolutional nonparametric Bayesian sparse ICA architecture for overcomplete feature learning from high-dimensional data. Our method utilizes an Indian buffet process prior to facilitate inference of the appropriate number of latent features under a hybrid variational inference algorithm, scalable to massive datasets. As we show, our model can be naturally used to obtain deep unsupervised hierarchical feature extractors, by greedily stacking successive model layers, similar to existing approaches. In addition, inference for this model is completely heuristics-free, thus, it obviates the need of tedious parameter tuning, which is a major challenge most deep learning approaches are faced with. We evaluate our method on several action recognition benchmarks, and exhibit its advantages over the state-of-the-art.
BibTeX
@inproceedings{chatzis2015nonparametric,
  author = {Sotirios P. Chatzis and Dimitrios I. Kosmopoulos},
  title = {{A Nonparametric Bayesian Approach toward Stacked Convolutional Independent Component Analysis}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2015},
  url = {https://openreview.net/forum?id=rkVto-z_-B},
}
Figure