Multimodal Data Visualization and Denoising with Integrated Diffusion

Manik Kuchroo, Abhinav Godavarthi, Alexander Tong, Guy Wolf, Smita Krishnaswamy

TL;DR

Integrated Diffusion is a novel geometric framework that combines noisy, multimodal datasets by adaptively denoising each modality both locally and globally before fusing them. By leveraging multiscale spectral denoising and spectral entropy to automatically tune diffusion steps, it creates a joint operator that significantly improves data visualization and feature denoising in complex systems like single-cell multi-omics.

What Problem Does This Paper Address?

Modern data science frequently handles parallel data types (modalities) collected from the same underlying system. For instance, in biomedical research, scientists can now simultaneously measure gene expression (RNA-seq) and chromatin accessibility (ATAC-seq) within a single cell.

However, integrating these modalities is notoriously difficult. Each data type comes with its own scale, sparsity, and unique noise characteristics. Naive approaches like concatenating features or using standard fusion techniques (such as alternating diffusion) often backfire: they inadvertently propagate noise from the messier modality into the cleaner one, washing out the true underlying geometry of the data. This paper addresses the challenge of combining these disparate data streams without letting modality-specific noise corrupt the joint representation.

Methods

The authors introduce Integrated Diffusion, a framework that constructs a robust joint Markov transition matrix (a diffusion operator) across multiple modalities. The workflow consists of three main stages:

  1. Neighborhood Reconstruction (Local Denoising): Using an algorithm called Multiscale Graph Denoising (MGD), the method recursively calculates local diffusion operators. This filters out high-frequency local noise while preserving the low-frequency eigenvectors that capture true data geometry.
  2. Global Denoising via Spectral Entropy: To determine how many diffusion steps ($t$) to take for each modality, the authors look at spectral entropy. By plotting spectral entropy against $t$, they locate an "elbow" where noise dimensions drop off and informative signals stabilize. This eliminates manual trial-and-error tuning.
  3. Operator Fusion: Instead of simply multiplying the transition matrices, the modalities are combined using a reduced ratio of their ideal, view-specific time steps ($J = P_1^{t_1}P_2^{t_2}$). This prevents an oversmoothing effect that would otherwise collapse distinct data points together.
Fig. 1. Workflow of integrated diffusion. We denoise each modality to local low frequency eigenvectors with multiscale spectral denoising. Next, we calculate and compare the intrin- sic dimensionality of each dataset via spectral entropy ratio to determine the ideal number of t-steps to place in each modality in an alternating random walk. The resulting diffusion operator can help denoise and visualize data.

Datasets

The method was validated on both synthetic benchmarks and real-world biological data:

  • Noisy MNIST: The classic handwritten digits dataset, modified by adding varying levels of global Gaussian noise to simulate two distinct modalities with asymmetric noise.
  • Artificial Tree Data: A synthetic manifold dataset with branch-specific local noise, designed to test the local denoising capabilities of the MGD algorithm.
  • Single-Cell Multi-Omic Data: A real-world dataset comprising 11,909 human blood cells, capturing parallel measurements of gene expression and chromatin accessibility.

Key Insights and Ideas

  • Noise is Asymmetric: In multimodal setups, one modality is often inherently messier than the other (e.g., ATAC-seq data is notoriously sparser and noisier than RNA-seq). Standard alternating diffusion blindly blends these spaces, degrading the quality of the cleaner modality.
  • Spectral Entropy as an Automated Tuning Knob: Treating the eigenvalues of a diffusion operator as a probability distribution allows spectral entropy to act as a direct measure of information spread. Finding the inflection point on this curve provides a principled way to automatically balance information content between different data types.
  • The Power of Alternating Random Walks: By carefully scaling the number of steps taken in each modality's graph before "hopping" to the next, the joint operator preserves unique structural information from both views while filtering out independent noise.

Key Results

  • Robustness to Global Noise: On the noisy MNIST classification task, Integrated Diffusion maintained high accuracy even as global noise increased, significantly outperforming standard alternating diffusion, canonical correlation analysis (CCA), and domain-transfer autoencoders.
  • Preserving Manifold Geometry: On the artificial tree dataset, the method achieved the highest DeMAP scores (a metric measuring how well embedding distances correlate with ground-truth geodesic distances), proving its superior ability to handle local, branch-specific noise.
  • Enhanced Biological Resolution: When applied to the 11,909 blood cells, Integrated Diffusion produced a PHATE visualization with sharper, more distinct cell-type boundaries. Crucially, it successfully recovered hidden cross-modality interactions (mutual information between a gene's accessibility and its expression) across 18,659 genes, outperforming all baseline methods.
Fig. 4. Application of integrated diffusion to multimodal single cell data. A) Visualization of gene expression and chromatin accessibility manifolds as well as B) alternating and integrated manifolds via PHATE. Points colored by annotated cell type. C) Visualization of spectral entropy of each modality. D) Mutual information between the expression of a gene and it’s accessibility across differing denoising strategies. E) Average mutual information for differing denoising strategies across all gene expression-gene accessibility pairs.

Limitations and Future Work

While the authors note that their flexible framework is conceptually extendable to more than two modalities, the current mathematical formulation and experimental validations are restricted to dual-modality integration. Additionally, the paper does not explicitly discuss the computational complexity or scaling bottlenecks that might arise when applying this hierarchical graph-based approach to massive, million-cell datasets.

Summary updated

Abstract

We propose a method called integrated diffusion for combining multimodal datasets, or data gathered via several different measurements on the same system, to create a joint data diffusion operator. As real world data suffers from both local and global noise, we introduce mechanisms to optimally calculate a diffusion operator that reflects the combined information from both modalities. We show the utility of this joint operator in data denoising, visualization and clustering, performing better than other methods to integrate and analyze multimodal data. We apply our method to multi-omic data generated from blood cells, measuring both gene expression and chromatin accessibility. Our approach better visualizes the geometry of the joint data, captures known cross-modality associations and identifies known cellular populations. More generally, integrated diffusion is broadly applicable to multimodal datasets generated in many medical and biological systems.
BibTeX
@inproceedings{kuchroo2021multimodal,
  author = {Manik Kuchroo and Abhinav Godavarthi and Alexander Tong and Guy Wolf and Smita Krishnaswamy},
  title = {{Multimodal Data Visualization and Denoising with Integrated Diffusion}},
  booktitle = {MLSP},
  year = {2021},
  url = {https://openreview.net/forum?id=HSe4tpmOSe5},
}
Figure