mixedCCA: Sparse Canonical Correlation Analysis for High-Dimensional Mixed Data

Grace Yoon, Irina GaynanovaOpenalex Unknown 2020 · DOI · OpenAlex

Abstract

A semi-parametric approach for sparse canonical correlation analysis (CCA) designed to handle mixed data types, including continuous, binary, and truncated continuous variables. The package provides bridge functions to connect Kendall's tau to latent correlation under the Gaussian copula model. These methods are detailed in Yoon, Carroll, and Gaynanova (2020) and Yoon, Mueller, and Gaynanova (2021).

Introduction

Canonical Correlation Analysis (CCA) is a cornerstone of multivariate statistics, identifying linear combinations of two sets of variables that are maximally correlated. However, traditional CCA faces significant hurdles in modern data science: it typically assumes variables are continuous and normally distributed, and it becomes unstable in high-dimensional settings where the number of variables ($p$) exceeds the sample size ($n$).

The mixedCCA package addresses these challenges by providing a robust semi-parametric framework. It allows researchers to uncover sparse, interpretable relationships between disparate data types—such as binary indicators and zero-inflated gene expression—without the strict assumption of multivariate normality.

Methodology

The core of mixedCCA is the Gaussian copula model, which assumes that observed mixed data arises from an underlying (latent) multivariate normal distribution.

Latent Correlation Estimation

Instead of standard Pearson correlation, the package utilizes Kendall’s tau, a rank-based measure robust to non-normality. To map the observed Kendall’s tau back to the latent correlation of the underlying Gaussian distribution, the package employs mathematical "bridge functions."

Parameters for the estimateR function, highlighting support for mixed data types and approximation methods.

As shown in the parameters for estimateR, the package offers an "original" optimization method and a faster "approx" (multilinear approximation) method. The approximation method is essential for high-dimensional applications where the computational cost of bridge functions can be prohibitive.

Sparse Optimization and Parameter Selection

To handle high-dimensional data, mixedCCA applies sparsity-inducing penalties to the canonical vectors. Tuning parameter selection is guided by the Bayesian Information Criterion (BIC), offering two specialized options: BIC1 for variable selection and BIC2 for prediction accuracy.

Parameters for the mixedCCA function, including options for BIC-based parameter selection and convergence controls.

Implementation and Key Insights

The package provides a comprehensive pipeline for mixed-data analysis. The estimateR function handles the estimation of latent correlation matrices, while the primary mixedCCA function outputs the estimated canonical directions ($w_1, w_2$) and the resulting canonical correlation.

A key insight of this approach is the power of the latent space. By treating binary and continuous variables on equal footing within a latent Gaussian framework, mixedCCA avoids the biases inherent in forcing categorical data into standard linear models. Furthermore, the inclusion of the nearPD utility ensures that estimated correlation matrices remain positive definite, a common issue in high-dimensional estimation.

To facilitate benchmarking, the package includes a GenerateData function. This allows users to create synthetic datasets with specific correlation structures (e.g., autocorrelation or block correlation) and mixed data types, providing a controlled environment to test the algorithm's performance against known ground truths.

Conclusion and Future Directions

mixedCCA offers a robust and efficient solution for Sparse CCA in the presence of mixed data. While powerful, the method currently relies on the Gaussian copula assumption. Future work may explore extensions to non-Gaussian copulas or broader data types, such as multi-level nominal or ordinal data. Additionally, as dimensionality continues to grow, further optimizations in matrix decomposition will be essential for scaling to extremely large datasets.

Summary updated

Abstract

Semi-parametric approach for sparse canonical correlation analysis which can handle mixed data types: continuous, binary and truncated continuous. Bridge functions are provided to connect Kendall's tau to latent correlation under the Gaussian copula model. The methods are described in Yoon, Carroll and Gaynanova (2020) &lt;<a href="https://doi.org/10.1093%2Fbiomet%2Fasaa007" target="_top">doi:10.1093/biomet/asaa007</a>&gt; and Yoon, Mueller and Gaynanova (2021) &lt;<a href="https://doi.org/10.1080%2F10618600.2021.1882468" target="_top">doi:10.1080/10618600.2021.1882468</a>&gt;.
BibTeX
@inproceedings{yoon2020mixedcca,
  author = {Grace Yoon and Irina Gaynanova},
  title = {{mixedCCA: Sparse Canonical Correlation Analysis for High-Dimensional Mixed Data}},
  booktitle = {Openalex Unknown},
  year = {2020},
  doi = {10.32614/cran.package.mixedcca},
}
Figure