Abstract: Unsupervised two-view learning, or detection of dependencies between two paired data sets, is typically done by some variant of canonical correlation analysis (CCA). CCA searches for a linear projection for each view, such that the correlations between the projections are maximized. The solution is invariant to any linear transformation of either or both of the views; for tasks with small sample size such flexibility implies overfitting, which is even worse for more flexible nonparametric or kernel-based dependency discovery methods. We develop variants which reduce the degrees of freedom by assuming constraints on similarity of the projections in the two views. A particular example is provided by a cancer gene discovery application where chromosomal distance affects the dependencies between gene copy number and activity levels. Similarity constraints are shown to improve detection performance of known cancer genes.
Introduction
In high-dimensional data analysis, particularly within biomedicine, researchers frequently encounter the "small $n$, large $p$" problem. This occurs when the number of samples (e.g., patients) is significantly smaller than the number of features (e.g., genes). Identifying meaningful relationships between two different "views" of the same items—such as gene copy numbers and gene expression levels—is a fundamental task.
The standard approach for this is Canonical Correlation Analysis (CCA). However, CCA's inherent flexibility is a double-edged sword. Because it is invariant to linear transformations, it can easily overfit to noise in small datasets, "hallucinating" correlations that do not exist. This is particularly problematic in genomics, where we expect a direct, similar relationship between a gene's physical copy number and its activity level.
Methodology: Similarity-constrained CCA (SimCCA)
To mitigate overfitting, we introduce SimCCA, which regularizes the search for dependencies by forcing projection vectors to be similar. We propose two primary variants:
- Correlation-based SimCCA: This method introduces a transformation matrix $T$ such that the projection vector for the second view, $v_y$, is related to the first, $v_x$, by $v_y = T v_x$. By constraining $T$ (e.g., to an identity matrix), we ensure the views are projected similarly, reducing the degrees of freedom.
- Probabilistic SimCCA (pSimCCA): A Bayesian extension that assumes both views are generated from a shared latent variable $z$. It incorporates a prior distribution on $T$ that penalizes deviations from similarity, allowing the model to handle noise more robustly.
These methods are applied using a sliding window approach, analyzing small neighborhoods of genes along the chromosome to detect localized dependencies.
Performance Evaluation
We validated our approach using a gastric cancer dataset consisting of 51 samples and 5,596 genes. Accuracy was measured against an expert-curated list of 59 known gastric-cancer-associated genes.
As shown in Figure 4, SimCCA significantly outperforms standard CCA and other probabilistic variants. While standard CCA's performance collapses as the window size (dimensionality) increases, SimCCA remains stable, achieving a maximum AUC of 0.79 with an optimal window size of 15 genes.
Biological Validation: The 17q Region
The practical utility of SimCCA is demonstrated by its ability to highlight biologically relevant regions. The model successfully identified the 17q12-q21 chromosomal region, which contains the ERBB2 gene, a critical target for cancer therapies like Herceptin.
Figure 2 illustrates how the SimCCA dependency score peaks in areas where both expression and amplification signals show correlated activity. Among the top 100 dependencies identified, 30% were known cancer genes, suggesting that the remaining 70% are high-probability candidates for future research.
Conclusion
By incorporating similarity as a prior, SimCCA effectively regularizes dependency detection in high-dimensional, low-sample-size scenarios. This approach reduces overfitting and improves the identification of functional biological relationships. Future work may explore non-linear extensions and the application of "soft" constraints to further refine dependency discovery across diverse paired data types.
Summary updated