Abstract
Learning representations of two views of data such that the resulting representations are highly linearly correlated is a fundamental challenge in multi-view learning. This paper introduces the Canonical Correlation Guided Deep Neural Network (CCDNN), a framework that merges multivariate analysis (MVA) with deep learning. Unlike traditional methods such as Deep CCA (DCCA), which maximize correlation as a primary objective, CCDNN treats canonical correlation as a constraint. This approach preserves the ability to learn correlated representations while focusing on specific engineering tasks like reconstruction, classification, and prediction. A key innovation is the "redundancy filter," designed to reduce information overlap between views. Experimental results on MNIST, industrial fault diagnosis, and remaining useful life (RUL) prediction datasets demonstrate that CCDNN outperforms existing methods in both accuracy and denoising performance.
Introduction
In multi-view learning, the goal is to extract meaningful features from multiple data sources—such as different sensor types on a single machine. The standard tool for this has been Canonical Correlation Analysis (CCA). However, traditional CCA and its deep variants (DCCA) often suffer from task misalignment and redundancy. By focusing solely on maximizing correlation, these models may ignore unique, task-relevant signals in favor of shared noise. CCDNN addresses these issues by reimagining correlation as a guide rather than the end goal.
Methodology
The CCDNN architecture is built on three pillars:
- Deep Feature Extractors: Parallel neural networks (e.g., CNNs or GRUs) map raw multi-view inputs into a latent space ($z_1, z_2$).
- CCA Constraint Layer: Instead of a correlation-based loss, CCDNN solves the CCA problem as a mathematical constraint during the forward pass, calculating linear mappings ($J, L$) and correlation coefficients without adding trainable parameters.
- Redundancy Filter (RF): This layer uses the CCA parameters to subtract the redundant, correlated components from the signals. The resulting residuals ($r_1, r_2$) contain the unique information from each view, which is then concatenated for downstream tasks.
Task-Specific Implementations
The framework is highly flexible and can be adapted for various engineering objectives. The authors demonstrate its utility across three primary tasks: image reconstruction, fault diagnosis (classification), and RUL prediction (regression).
1. Reconstruction Task
Using a modified MNIST dataset with noisy and rotated views, CCDNN employs decoders to reconstruct the original images from the filtered latent representations. By removing redundant noise through the RF layer, CCDNN achieved a Mean Squared Error (MSE) of 0.10, significantly outperforming DCCA (0.53) and DCCAE (0.19).
2. Fault Diagnosis
Tested on the SEU Bearing dataset, the model uses vibration and torque data to classify gearbox faults. The CCDNN architecture reached a diagnostic accuracy of 97.79%, showing particular strength in scenarios with limited training data compared to standard CNNs.
3. Remaining Useful Life (RUL)
For the NASA C-MAPSS turbofan engine dataset, CCDNN was integrated with Gated Recurrent Units (GRUs). The CCA-guided features allowed the model to predict the remaining lifespan of engines with lower error rates than standalone temporal models, proving that the correlation constraint helps isolate critical degradation signals.
Conclusion
CCDNN represents a significant step in integrating classical multivariate analysis with modern deep learning. By treating correlation as a constraint and introducing a redundancy filter, the model effectively balances shared information with task-specific features. While the current implementation relies on a linear redundancy filter, the framework's modularity allows for future extensions into fully nonlinear filtering and deeper residual architectures.
Summary updated