Conditional Hierarchical Bayesian Tucker Decomposition

Adam Sandler, Diego Klabjan, Yuan Luo

Introduction

Analyzing genetic data often involves navigating a "haystack" of millions of variants, where the data is characterized by extreme sparsity and high dimensionality. Researchers from Northwestern University have introduced Conditional Hierarchical Bayesian Tucker (HBT) Decomposition, a framework designed to organize these sparse features into biologically meaningful, hierarchical "topics."

The Challenge of Sparse Genomic Data

Genetic datasets typically feature tens of thousands of genes (features) across only a few thousand patients (observations). Most patients possess only a fraction of possible mutations, resulting in a data matrix dominated by zeros. Standard supervised learning often struggles in this environment due to the high feature-to-sample ratio. While traditional methods focus on individual gene markers, they frequently overlook the broader biological pathways—groups of genes working in concert—that drive complex conditions like cancer or Autism Spectrum Disorder (ASD).

Methodology: HBT Decomposition

The HBT model integrates two core concepts from machine learning and statistics:

  1. Tucker Decomposition: This method treats data as a multi-dimensional tensor (e.g., Patients × Genes × Pathways). By utilizing a "core tensor," it models complex interactions across different dimensions more effectively than simpler matrix factorizations.
  2. Hierarchical Topic Modeling: Drawing from Natural Language Processing, the model employs nested Chinese Restaurant Processes (nCRP) and Pachinko Allocation Machines (PAM). This creates a tree-like structure where top-level topics represent broad genetic signatures, and lower levels capture specific, rare sub-groups.

The "Conditional" aspect refers to analyzing feature variables (genes/pathways) conditioned on the sample variable (the patient). To ensure computational feasibility, the authors derived a collapsed Gibbs sampler to efficiently estimate parameters.

Plate notation for the Conditional Hierarchical Bayesian Tucker model, illustrating the relationships between priors, latent topics, and observed features.

Mathematical Framework and Notation

The model's variables are defined to map clinical and biological entities to a probabilistic framework. For instance, $x$ represents the patient index, while $\mathbf{y}$ represents feature variables like genes and pathways. The following table details the notation used throughout the HBT framework.

Definitions of variables and parameters used in the HBT framework, mapping mathematical symbols to biological examples.

Experimental Validation

The model was tested on three distinct datasets:

  • The Cancer Genome Atlas (TCGA): 3,037 patients across four cancer types, involving over 7,800 genes and 1,600 pathways.
  • National Database for Autism Research (NDAR): Genetic variant counts from 3,408 siblings.
  • Reuters-21578 (R8): A text dataset used for benchmarking traditional topic modeling.

Key Insights and Results

  • Enhanced Coherence: The hierarchical structure significantly improved topic coherence. On cancer data, HBT outperformed baselines like hierarchical LDA (hLDA) and CP decomposition in three out of four coherence metrics. Notably, it improved "UMass pathway coherence" by 34.52%.
  • Scalability: The Tucker core allows the model to scale better to multiple dimensions than CP-based models. Training times remained competitive even as the number of topics increased.
  • Biological Interpretability: In ASD data, the model identified topics related to immune function, neurotransmission, and lipid homeostasis—all recognized risk factors in autism research.

Conclusion and Future Directions

While HBT excels at identifying biologically relevant patterns and providing an interpretable "map" for clinicians, it faces a trade-off in classification accuracy. When using discovered topics for cancer classification, accuracy (55-69%) was lower than using raw gene counts (83%), suggesting that the abstraction process may discard some high-resolution information necessary for prediction. Future work may explore hybrid models that combine hierarchical and independent dimensions to capture finer nuances in the genetic code.

Summary updated

Abstract

We develop methods for reducing the dimensionality of large data sets, common in biomedical applications. Learning about patients using genetic data often includes more features than observations, which makes direct supervised learning difficult. One method of reducing the feature space is to use latent Dirichlet allocation to group genetic variants in an unsupervised manner. Latent Dirichlet allocation describes a patient as a mixture of topics corresponding to genetic variants. This can be generalized as a Bayesian tensor decomposition to account for multiple feature variables. Our most significant contributions are with hierarchical topic modeling. We design distinct methods of incorporating hierarchical topic modeling, based on nested Chinese restaurant processes and Pachinko Allocation Machine, into Bayesian tensor decomposition. We apply these models to examine patients with one of four common types of cancer (breast, lung, prostate, and colorectal) and siblings with and without autism spectrum disorder. We linked the genes with their biological pathways and combine this information into a tensor of patients, counts of their genetic variants, and the genes' membership in pathways. We find that our trained models outperform baseline models, with respect to coherence, by up to 40%.
BibTeX
@inproceedings{sandler2025conditional,
  author = {Adam Sandler and Diego Klabjan and Yuan Luo},
  title = {{Conditional Hierarchical Bayesian Tucker Decomposition}},
  booktitle = {CAI},
  year = {2025},
  doi = {10.1109/cai64502.2025.00079},
}
Figure