Co-regularized multiview nonnegative matrix factorization with correlation constraint for representation learning

Weihua Ou, Fei Long, Yi Tan, Shujian Yu, Pengpeng WangMultimedia Tools and Applications 2018 · DOI

Abstract

In real-world applications, data is often collected from multiple sources or "views," such as different camera angles, modalities, or feature sets. While multi-view data provides richer information, it also introduces challenges like noise sensitivity and quality imbalance between views. This paper proposes a novel framework that combines Co-regularized Nonnegative Matrix Factorization (NMF) with a correlation constraint. By enforcing pairwise similarity between view-specific representations and maximizing their correlation with a shared latent consensus, the model produces robust and discriminative features. Experimental results on diverse datasets demonstrate the method's superiority in handling noisy and occluded data for clustering and classification tasks.

Introduction

Modern machine learning often deals with objects described by multiple distinct feature sets. For instance, a person might be identified by their face, iris, and fingerprint, or a document might be represented by text in different languages. These are referred to as "multiple views."

Examples of multi-view data sources including 3D perspectives, object attributes, biometrics, and multilingual text.

Despite the potential for improved performance, two major hurdles persist in multi-view learning:

  1. Noise Sensitivity: Real-world data is frequently corrupted by noise (e.g., sensor errors or occlusions). Standard methods often fail to distinguish meaningful patterns from this noise.
  2. Quality Imbalance: Different views vary in their informative value. A low-quality or noisy view can degrade the performance of the entire system if not handled carefully.

The authors address these issues by developing a latent representation that captures consistent information across views while remaining robust to view-specific noise.

Methodology

The core of the proposed approach is built upon Nonnegative Matrix Factorization (NMF), which decomposes a data matrix into basis and coefficient matrices under non-negativity constraints. To enhance this for the multi-view setting, the authors introduce two key components:

Pairwise Co-regularization

Instead of forcing all views to immediately align with a single global template, the model encourages pairs of views to be similar. This "soft" constraint allows the system to accommodate views of varying quality, preventing a single highly noisy view from corrupting the entire consensus.

Correlation Constraint

To ensure the final representation is meaningful, the model maximizes the correlation between individual view representations ($V^{(i)}$) and a common latent consensus ($V^*$). This forces the model to extract the underlying structure shared across all perspectives.

The proposed multi-view NMF framework pipeline, from feature extraction to alternating optimization for a consensus representation.

The optimization problem is solved using an alternative update algorithm, which iteratively refines the representations while preserving non-negativity.

Experimental Evaluation

The researchers evaluated their method against several baselines, including Feature Concatenation (FC) and standard Multi-view NMF, across four diverse datasets:

  • ORL Face Dataset: 400 grayscale images using raw pixels and Local Binary Patterns (LBP) as views.
  • CMU-PIE Dataset: Face images with varying illumination. Robustness was tested by adding random block occlusions.
  • UCI Handwritten Digit Dataset: 2,000 samples with salt & pepper noise (5% to 25%) to test noise resistance.
  • OuluVS Dataset: A lipreading dataset representing complex multivariate time series.

Key Results

The proposed method consistently outperformed competitors:

  • Accuracy (ACC): On the ORL dataset, the method achieved 49.9% accuracy, compared to 46.3% for the next best method (CoNMF).
  • Noise Resistance: On the UCI Digit dataset with 25% noise, the method maintained significantly higher Accuracy and Normalized Mutual Information (NMI) scores, whereas other methods showed sharp performance declines.
  • Occlusion Handling: On the CMU-PIE dataset, the method achieved an average ACC of 28.5%, nearly 3% higher than standard Multi-NMF.

Conclusion

This paper presents a robust approach to multi-view representation learning by leveraging co-regularization and correlation constraints. The "aha moment" lies in exploiting both consistent and complementary information, allowing the model to effectively "ignore" view-specific noise. Future work may involve incorporating supervision (label information) to further enhance discriminative power and addressing the challenge of incomplete views where data points are missing from certain perspectives.

Summary updated

BibTeX
@article{ou2018co,
  author = {Weihua Ou and Fei Long and Yi Tan and Shujian Yu and Pengpeng Wang},
  title = {{Co-regularized multiview nonnegative matrix factorization with correlation constraint for representation learning}},
  journal = {Multimedia Tools and Applications},
  year = {2018},
  doi = {10.1007/s11042-017-4926-0},
}
Figure