Multimodal Fusion Using Sparse Cca For Breast Cancer Survival Prediction

Vaishnavi Subramanian, Tanveer F. Syeda-Mahmood, Minh N. Do

Introduction

Cancer is a complex, multi-scale disease that manifests across various biological levels, from molecular gene expressions to macro-level tissue morphology. To improve clinical outcomes, such as predicting patient survival, it is essential to integrate these disparate data modalities. However, "fusing" high-dimensional genomic data with histology images presents significant challenges. Standard methods like Canonical Correlation Analysis (CCA) are effective at identifying shared information across modalities but often fail to account for the intricate internal dependencies within a single modality—such as gene-gene interaction networks. Furthermore, biological data is frequently noisy and high-dimensional, necessitating sparse solutions that select only the most relevant features.

The GCCA Framework

To address these challenges, this research proposes a Graph-based Sparse CCA (GCCA) framework. This approach explicitly models the internal structure of each data source while maximizing the shared information between them.

Overview of the multimodal fusion framework using graph-structured Sparse CCA.

The framework consists of several key components:

  1. Intra-modality Graph Regularization: For each modality, internal dependencies are represented as a graph. A graph Laplacian matrix ensures that features closely related in the graph (e.g., proteins in the same pathway) receive similar weights in the embedding.
  2. Bi-convex Optimization: The model optimizes an objective function that maximizes inter-modality correlation while penalizing embeddings that violate the graph structure or lack sparsity.
  3. Multi-dimensional Embeddings (K-GCCA): Using Hotelling’s deflation scheme, the model iteratively extracts $K$ orthogonal embedding directions. This captures a richer, multi-dimensional representation ($Z_e$) compared to standard 1D correlation vectors.
  4. Survival Prediction: The resulting compact embeddings are fed into a Random Forest classifier to predict one-year survival.

Experimental Setup

The framework was evaluated using the TCGA-BRCA (The Cancer Genome Atlas Breast Invasive Carcinoma) dataset:

  • Imaging Data: Histology slides were processed via nuclei segmentation to extract 213 features related to shape, texture, and area.
  • Genomic Data: Gene expression levels for the 500 to 3,000 most variant genes were utilized.
  • Prior Knowledge: The STRING database of protein-protein interactions provided the "prior knowledge" graph for the genomic modality, though data-driven graphs were also evaluated.

Results and Discussion

The GCCA approach demonstrated superior performance in identifying meaningful relationships between genomics and histology compared to standard Sparse CCA (SCCA).

Correlation coefficients across different numbers of genomic features (p) for SCCA and GCCA variants.

As shown in the results, 1-GCCA consistently achieved higher correlation coefficients (up to 0.56) compared to standard SCCA, which ranged between 0.36 and 0.46. Notably, the GCCA method remained robust as the number of genomic features ($p$) increased, whereas standard SCCA performance was more volatile.

In the downstream task of survival prediction, the 100-GCCA embedding module achieved an average F1 score of 58.69% for one-year survival classification. The inclusion of graph structures acted as a powerful regularizer, preventing the model from picking up spurious correlations and ensuring that selected features were biologically grounded.

Conclusion and Future Work

Graph-structured Sparse CCA provides a robust method for multimodal fusion in cancer research. By respecting the internal architecture of biological data, it produces higher-quality embeddings that improve predictive performance. Future research will explore more sophisticated deflation schemes for high-dimensional embeddings and evaluate the generalizability of this approach across other cancer types. Additionally, integrating this linear embedding module into end-to-end deep learning architectures represents a promising direction for further performance gains.

Summary updated

BibTeX
@inproceedings{subramanian2021multimodal,
  author = {Vaishnavi Subramanian and Tanveer F. Syeda-Mahmood and Minh N. Do},
  title = {{Multimodal Fusion Using Sparse Cca For Breast Cancer Survival Prediction}},
  booktitle = {ISBI},
  year = {2021},
  doi = {10.1109/isbi48211.2021.9434033},
}
Figure