DCID: Deep Canonical Information Decomposition

Alexander Rakowski, Christoph Lippert

In biomedical research, scientists often analyze pairs of scalar measurements—such as a patient’s Body Mass Index (BMI) and the volume of their hippocampus—to understand underlying biological relationships. While simple correlations can be calculated, these aggregate values often mask a complex web of shared and unique biological signals. A new framework, Deep Canonical Information Decomposition (DCID), addresses this by isolating the specific information shared between two traits using high-dimensional auxiliary data, such as MRI scans.

The Challenge of Univariate Shared Information Retrieval

The core problem is identifying a shared signal $Z$ between two target variables $Y_1$ and $Y_2$ when those targets also contain individual signals $Z_1$ and $Z_2$. Traditional methods struggle in this "univariate" context:

  1. Canonical Correlation Analysis (CCA): While effective for multivariate data, CCA yields only a trivial correlation when applied to two single numbers.
  2. Multi-Task Learning (MTL): These models are optimized for prediction accuracy rather than the pure recovery of underlying latent signals. They often conflate shared and individual information.
Graphical model illustrating how a shared signal (Z) and individual signals (Z1, Z2) generate both scalar targets (Y) and high-dimensional observations (X).

The DCID Framework

DCID overcomes these limitations by "unaggregating" scalar targets back into a high-dimensional space. The model architecture follows a two-stage process:

1. Feature Extraction and Multivariate Decomposition

Two separate Deep Neural Networks (DNNs), $f_1$ and $f_2$, are trained to predict $Y_1$ and $Y_2$ using high-dimensional observations $X$. Instead of using the final scalar predictions, DCID extracts the outputs from the penultimate layers. These serve as multivariate representations ($B^1, B^2$) of the targets, capturing the rich information contained in $X$ that is relevant to $Y$.

2. CCA Integration

Classical CCA is then performed on these multivariate features. The most correlated components represent the shared signal $\hat{Z}$, effectively isolating the common thread between the two traits.

The DCID model architecture: mapping scalar targets to multivariate latent representations to enable decomposition via CCA.

Evaluating Recovery: The ICM Score

To validate the model, the authors introduced the ICM score, which evaluates three dimensions of the learned feature $\hat{Z}$:

  • Informativeness: The ability to predict the true $Z$ from the estimate.
  • Completeness: Whether the true $Z$ is sufficient to explain the estimate.
  • Minimality: The successful exclusion of individual signals $Z_1$ and $Z_2$.

Key Results and Insights

Prediction vs. Recovery

A critical insight from this work is that high predictive performance (high $R^2$) does not guarantee accurate signal recovery. Standard MTL models often "leak" individual signals into the shared feature space. In synthetic tests (Shapes3D), DCID achieved an ICM score of 0.62, significantly outperforming MTL and MTFL baselines, which scored approximately 0.18–0.19.

Biomedical Application: UK Biobank

Applying DCID to the UK Biobank dataset—using brain MRI scans as $X$ and Body Fat Mass (BFM) alongside brain region volumes as $Y$—yielded powerful results. The shared signals extracted by DCID acted as "surrogate" variables for obesity that correlated with brain changes up to 8 times more strongly than raw measurements. For instance, the correlation between BFM and ventricle volume was -0.02, whereas the DCID-extracted shared signal showed a correlation of -0.17.

Conclusion and Future Directions

DCID demonstrates that by leveraging high-dimensional data, we can apply multivariate tools to univariate problems, providing a clearer picture of shared biological mechanisms. However, the method is a decomposition tool, not a causal one; it identifies shared information without determining the direction of influence. Future work will explore the model's behavior in "strong-signal" regimes where the shared component is extremely dominant, as well as its application to other high-dimensional data types beyond imaging.

Summary updated

BibTeX
@inproceedings{rakowski2023dcid,
  author = {Alexander Rakowski and Christoph Lippert},
  title = {{DCID: Deep Canonical Information Decomposition}},
  booktitle = {ECML/PKDD},
  year = {2023},
  doi = {10.1007/978-3-031-43415-0\_2},
}

Loading related content...

Figure