Co-Clustering Multi-View Data Using the Latent Block Model

Joshua Tobin, Michaela Black, James Ng, Debbie Rankin, Jonathan Wallace, Catherine Hughes, Leane Hoey, Adrian Moore, Jinling Wang, Geraldine Horigan, Paul Carlin, Helene McNulty, Anne M. Molloy, Mimi ZhangComput. Stat. Data Anal 2024

TL;DR

This paper introduces the Multi-View Latent Block Model (MVLBM), a novel co-clustering approach designed to analyze datasets with multiple, distinct feature sets (views). It extends the traditional Latent Block Model to handle these multi-view scenarios, enabling simultaneous clustering of observations and features across different data modalities while accounting for dependencies between views.

What Problem Does This Paper Address?

Modern datasets often consist of multiple "views" of the same observations, such as gene expression, methylation data, and clinical outcomes for a set of patients. Traditional co-clustering methods, like the Latent Block Model (LBM), are designed for single data matrices and struggle to effectively integrate information from these disparate views. This limitation hinders the ability to uncover complex relationships and patterns that span across different data modalities. The authors address this by developing a method that can jointly co-cluster multi-view data, allowing for a more comprehensive understanding of the underlying structure.

Methods

The core of the paper is the Multi-View Latent Block Model (MVLBM). Key methodological contributions include:

  • Extending the LBM: The MVLBM assumes that each view marginally follows an LBM, but crucially, it models the dependency between the row cluster memberships across different views using a cluster membership matrix ($\pi$).
  • Stochastic EM Algorithm (SEM-Gibbs): To handle the computational complexity of estimating the MVLBM, the authors adapt a SEM-Gibbs algorithm. This approach integrates Gibbs sampling for latent variable estimation and a weighted maximum likelihood approach for parameter updates, making it more robust to initialization issues than standard EM.
  • Hypothesis Testing for Independence: A novel hypothesis test is introduced to assess whether row clusterings in two views are independent. This test helps in deciding whether to retain dependencies between views in the MVLBM, guiding the model selection process.
  • Sparsity-Inducing Log Penalty: To manage the exponential growth of the model space and prevent empty clusters, a penalized likelihood approach is used. This penalty encourages sparsity in the row cluster memberships, effectively removing less informative clusters.
  • ICL Criterion for Model Selection: The Integrated Completed Likelihood (ICL) criterion is adapted for model selection, helping to determine the optimal number of row and column clusters in each view.

Datasets

The authors evaluate their method on:

  • Synthetic Datasets: Multiple simulated datasets are generated to test various aspects of the MVLBM, including its ability to detect dependencies, estimate parameters, handle missing data, and perform model selection under different configurations of data types (nominal, ordinal, continuous, count) and cluster structures.
  • Chronic Lymphocytic Leukaemia (CLL) Dataset: A real-world multi-view dataset combining drug response, DNA methylation, transcriptome profiling (mRNA), and somatic mutation status is used to demonstrate the MVLBM's practical utility and its ability to uncover clinically relevant insights.

Key Insights and Ideas

  • Bridging the Gap in Multi-View Clustering: The MVLBM provides a principled way to co-cluster data from multiple sources, moving beyond single-view analyses.
  • Dependency Matters: Explicitly modeling the dependency between row cluster memberships across views is crucial for uncovering richer patterns than simply analyzing each view independently.
  • Computational Challenges are Real: The complexity of multi-view co-clustering necessitates sophisticated inference algorithms like SEM-Gibbs and careful model selection strategies.
  • Sparsity is Beneficial: The log penalty effectively prunes the model space, leading to more interpretable and parsimonious clusterings.
  • Real-World Impact: The application to the CLL dataset highlights the MVLBM's potential to identify key biomarkers and relationships in complex biological data.

Key Results

  • The hypothesis testing procedure effectively detects dependencies between row clusterings, controlling Type I error and exhibiting good power.
  • The SEM-Gibbs algorithm demonstrates accurate parameter estimation, even with limited cluster separation or missing data. Mean absolute errors for estimated parameters are consistently low.
  • The ICL criterion, combined with a greedy search strategy, effectively selects the correct number of row and column clusters.
  • The sparsity-inducing log penalty successfully removes redundant clusters, leading to more parsimonious models.
  • On the CLL dataset, the MVLBM identified significant relationships between views and uncovered clinically relevant insights, particularly regarding IGHV mutation status and its association with specific gene mutations and methylation patterns. The method achieved an ARI score of 0.79 for the IGHV mutation status.

Limitations and Future Work

  • Computational Cost: While the SEM-Gibbs algorithm is an improvement, multi-view co-clustering remains computationally intensive, especially for very large datasets or a high number of clusters.
  • Model Space Exploration: Exhaustively exploring the entire model space for optimal cluster numbers is computationally infeasible. The proposed greedy search and iterative refinement are practical but may not guarantee finding the global optimum.
  • Future Work: The authors suggest extending the MVLBM to incorporate other data types like functional and text data. They also plan to explore alternative schemes for inducing sparsity in the joint space of clusters.

Summary updated

Abstract

The Latent Block Model (LBM) is a prominent model-based co-clustering method, returning parametric representations of each block cluster and allowing the use of well-grounded model selection methods. The LBM, while adapted in literature to handle different feature types, cannot be applied to datasets consisting of multiple disjoint sets of features, termed views, for a common set of observations. In this work, we introduce the multi-view LBM, extending the LBM method to multi-view data, where each view marginally follows an LBM. In the case of two views, the dependence between them is captured by a cluster membership matrix, and we aim to learn the structure of this matrix. We develop a likelihood-based approach in which parameter estimation uses a stochastic EM algorithm integrating a Gibbs sampler, and an ICL criterion is derived to determine the number of row and column clusters in each view. To motivate the application of multi-view methods, we extend recent work developing hypothesis tests for the null hypothesis that clusters of observations in each view are independent of each other. The testing procedure is integrated into the model estimation strategy. Furthermore, we introduce a penalty scheme to generate sparse row clusterings. We verify the performance of the developed algorithm using synthetic datasets, and provide guidance for optimal parameter selection. Finally, the multi-view co-clustering method is applied to a complex genomics dataset, and is shown to provide new insights for high-dimension multi-view problems.
BibTeX
@article{tobin2024co,
  author = {Joshua Tobin and Michaela Black and James Ng and Debbie Rankin and Jonathan Wallace and Catherine Hughes and Leane Hoey and Adrian Moore and Jinling Wang and Geraldine Horigan and Paul Carlin and Helene McNulty and Anne M. Molloy and Mimi Zhang},
  title = {{Co-Clustering Multi-View Data Using the Latent Block Model}},
  journal = {Comput. Stat. Data Anal.},
  year = {2024},
}
Figure