Dynamic visualization of high-dimensional data

Eric Sun, Rong Ma, James ZoubioRxiv 2022 · DOI

TL;DR

This paper introduces DynamicViz, a framework for creating dynamic visualizations of high-dimensional data. By generating multiple visualizations from bootstrapped data and aligning them, DynamicViz reveals the sensitivity of dimensionality reduction (DR) methods to data perturbations. A novel "variance score" quantifies this variability, offering a new way to assess visualization reliability and optimize DR algorithms.

What Problem Does This Paper Address?

High-dimensional data is ubiquitous in fields like genomics and imaging. Dimensionality reduction (DR) techniques are essential for visualizing this data in lower dimensions, enabling hypothesis generation and insight discovery. However, DR methods inherently introduce distortions, and static visualizations can mask the inherent variability and potential biases introduced by the DR algorithm or the data itself. This can lead to misinterpretations, such as incorrectly identifying clusters, missing connections between them, or misinterpreting data ordering. There's a critical need for methods that can reliably assess the quality and stability of these visualizations, going beyond simple concordance metrics.

Methods

The core of DynamicViz involves generating multiple DR visualizations from bootstrapped samples of the original data. These bootstrapped visualizations are then rigidly aligned to a reference visualization to ensure comparability. The aligned visualizations are presented dynamically through three modalities:

  • Interactive HTML: Allows users to explore the data with dynamic transitions, toggling points, and embedded metadata.
  • Animated GIF/AVI: Creates video or GIF files for easy embedding in presentations and workflows.
  • Stacked Visualization: Overlays all bootstrap visualizations with varying opacity to show density and robustness.

A key innovation is the variance score, which quantifies the variability of an observation's position across the bootstrapped visualizations. This score is calculated by measuring the average variance in pairwise Euclidean distances between an observation and its neighbors. The paper also proposes a decomposition of the variance score into components attributable to data stochasticity and DR algorithm stochasticity.

Datasets

The authors evaluated DynamicViz and the variance score on several real-world datasets, including:

  • Single-cell transcriptomic data from mouse subventricular zone.
  • Human genomics data from the 1000 Genomes Project.
  • Astronomical object data from the Sloan Digital Sky Survey.
  • Single-cell RNA-seq data from mouse embryonic stem cells (cell cycle phases).
  • Single-cell RNA-seq data from human bone marrow.
  • Single-cell RNA-seq data from gastrulation of the erythroid lineage.
  • Single-cell RNA-seq data from mouse pancreas (for RNA velocity analysis).
  • Spatial transcriptomics data from mouse primary motor cortex (MERFISH).
  • CITE-seq data (paired transcriptomic and surface proteomic) from human bone marrow.

Synthetic data from mixtures of Gaussian distributions was also generated to validate theoretical properties of the bootstrap and variance score.

Key Insights and Ideas

  • Dynamic visualizations reveal hidden instability: Static DR plots can present a single, potentially misleading, view. Dynamic visualizations, by showing multiple bootstrapped views, expose how sensitive the visualization is to small changes in the data or the DR algorithm's initialization.
  • Bridging connections are not always real: What appears as a connection between clusters in a static plot might be an artifact of the DR method. Dynamic visualization helps distinguish robust connections from spurious ones.
  • Variance score as a quality metric: The variance score provides a novel, observation-specific metric for visualization quality that captures variability. This is crucial because it accounts for both data noise and DR algorithm stochasticity, unlike many existing metrics.
  • Optimizing DR algorithms: The variance score can be used to select optimal hyperparameters for DR algorithms (e.g., perplexity in t-SNE) and even to choose the best DR algorithm for a given dataset, by minimizing the variance score.
  • Cross-modal consistency: The variance score shows correlation across different data modalities (e.g., transcriptomics vs. proteomics), suggesting it captures a fundamental aspect of biological variability that is reflected in DR visualizations.

Key Results

  • DynamicViz successfully diagnosed interpretative pitfalls in static DR visualizations, such as the stability of bridging connections and the robustness of clusters.
  • The variance score was shown to be correlated with existing quality metrics and also with biological variability (e.g., mean gene expression variance across replicates).
  • For t-SNE, the variance score was minimized at a perplexity of 160, leading to better visual separation and stability.
  • When comparing different DR algorithms on mouse SVZ data, t-SNE yielded the lowest mean variance score, suggesting it produced the most stable visualizations.
  • The variance score was found to be a good approximation of the global variance score even with a "random" neighborhood definition, making it computationally more feasible for large datasets.

Limitations and Future Work

  • Computational Cost: Generating many DR visualizations for bootstrapping can be computationally expensive, especially for datasets with many observations. The authors note that parallelization in DynamicViz can help mitigate this.
  • Algorithm Compatibility: The bootstrap sampling approach is directly applicable to DR algorithms that can handle duplicate observations. Extensions to methods that do not inherently support duplicates are mentioned as future work, including adding noise or subsampling.
  • Theoretical Guarantees: The theoretical guarantees of the bootstrap visualization are currently valid under specific regularity conditions and need to be proven for individual DR algorithms.
  • Future Directions: The authors suggest further research into handling duplicate observations more optimally and extending the theoretical framework to a broader range of DR algorithms and data modalities.

Summary updated

Abstract

Abstract Dimensionality reduction (DR) is commonly used to project highdimensional data into lower dimensions for visualization, which could then generate new insights and hypotheses. However, DR algorithms necessarily introduce distortions in the visualization and cannot faithfully represent all relations in the data. As such, there is a need for methods to assess the reliability of DR visualizations. Here we present DynamicViz, a framework for generating dynamic visualizations that capture the sensitivity of DR visualizations to perturbations in the data. DynamicVic can be applied to all commonly used DR methods. We show the utility of dynamic visualizations in diagnosing common interpretative pitfalls of static visualizations and extending existing single-cell analyses. We introduce the variance score to quantify the dynamic variability of observations in these visualizations. The variance score characterizes natural variability in the data and can be used to optimize DR algorithm implementations. We have made DynamicViz freely available to assist with the evaluation of DR visualizations.
BibTeX
@misc{sun2022dynamic,
  author = {Eric Sun and Rong Ma and James Zou},
  title = {{Dynamic visualization of high-dimensional data}},
  year = {2022},
  doi = {10.1101/2022.05.27.493785},
  howpublished = {bioRxiv},
}
Figure