Superheat: An R Package for Creating Beautiful and Extendable Heatmaps for Visualizing Complex Data

Rebecca Barter, Bin YuJournal of Computational and Graphical Statistics 2015 · arXiv

TL;DR

This paper introduces superheat, a new R package designed to create highly customizable and visually appealing heatmaps for complex, high-dimensional data. It extends traditional heatmaps by allowing the integration of additional data types like scatterplots, boxplots, and barplots, making it a powerful tool for exploratory data analysis and model assessment.

What Problem Does This Paper Address?

In today's data-rich world, extracting meaningful insights from massive, high-dimensional datasets is a significant challenge. Traditional visualization tools often struggle to scale effectively, leading to information overload or an inability to represent complex relationships. While heatmaps are a powerful technique for visualizing large datasets, especially in fields like bioinformatics, they are underutilized in broader data analysis. This paper aims to bridge this gap by providing a flexible and extendable heatmap solution that can handle diverse data types and reveal deeper insights.

Methods

The core contribution is the superheat R package, which builds upon the ggplot2 visualization infrastructure. It offers an intuitive interface for creating heatmaps while significantly enhancing their customizability and functionality. Key novel features include:

  • Multi-type Data Integration: superheat allows users to overlay additional visualizations directly onto the heatmap, such as scatterplots for response variables, boxplots for model results, and barplots for correlation information. This enables a more holistic view of the data.
  • Extensibility and Customization: The package provides extensive options for customizing color schemes, labels, grid lines, and adjacent plots, allowing for highly tailored visualizations.
  • Smoothing Functionality: For very large matrices, superheat offers a smoothing option that aggregates cells within clusters, making it easier to discern overall patterns.
  • Hierarchical Clustering Integration: The package seamlessly integrates with hierarchical clustering, displaying dendrograms alongside the heatmaps to illustrate data structure.

Datasets

The paper demonstrates the superheat package using three case studies based on publicly available data:

  1. Global Organ Transplantation Trends: Data from the WHO-ONT Global Observatory on Donation and Transplantation and the United Nations Development Program's Human Development Reports.
  2. Language Clusters: Google News word vectors (300-dimensional for 3 million words) and the NY Times headlines dataset.
  3. fMRI Brain Signals: Functional Magnetic Resonance Imaging (fMRI) data from a single individual, measuring brain responses to visual stimuli.

Key Insights and Ideas

  • Heatmaps as a Default Visualization: The paper argues that heatmaps, enhanced with additional information, can serve as a powerful default visualization method for a wide range of complex datasets, moving beyond their traditional use in bioinformatics.
  • Uncovering Heterogeneity: superheat excels at revealing heterogeneity within data. For instance, in the fMRI case study, it clearly identified distinct groups of voxels with differing predictive performance and sensitivity to image stimuli, a nuance that might be missed with simpler visualizations.
  • Contextualizing Word Embeddings: Visualizing word clusters using superheat provided a more intuitive understanding of semantic relationships and hierarchical structures within language data than standard low-dimensional projections.
  • Simultaneous Multi-Trend Visualization: The organ donation case study demonstrated how superheat can effectively display multiple trends (donation rates over time, HDI rankings, regional groupings) simultaneously without overplotting, offering a rich, uncluttered summary.

Key Results

The paper showcases the effectiveness of superheat through three case studies:

  • Organ Donation: superheat successfully visualized the relationship between organ donation rates and HDI rankings across countries, highlighting regional differences and trends over time.
  • Word Clusters: The package visualized cosine similarity matrices for word embeddings, revealing topic-specific clusters (e.g., political, legal, conflict) and their hierarchical relationships. It also allowed for the evaluation of cluster quality using silhouette widths.
  • fMRI Data: superheat identified two distinct groups of voxels ("sensitive" and "neutral") based on their response to visual stimuli and demonstrated that models for "sensitive" voxels performed significantly better. This revealed spatial patterns of voxel sensitivity within the visual cortex.

Limitations and Future Work

  • Scalability for Extremely Large Datasets: While superheat is designed for large datasets, the paper notes that for extremely massive matrices, the "graininess" of the raw heatmap might still be an issue, necessitating the use of smoothing or aggregation techniques.
  • Default Clustering: The default clustering algorithm is K-means, and the paper recommends users select the optimal number of clusters using methods like silhouette plots prior to implementation.
  • Future Work: The paper implicitly suggests that further exploration of superheat's capabilities in various scientific domains and the development of more advanced interactive features could be valuable. The availability of the superheat package on GitHub and detailed vignettes points towards ongoing development and community engagement.

Summary updated

Abstract

The technological advancements of the modern era have enabled the collection of huge amounts of data in science and beyond. Extracting useful information from such massive datasets is an ongoing challenge as traditional data visualization tools typically do not scale well in high-dimensional settings. An existing visualization technique that is particularly well suited to visualizing large datasets is the heatmap. Although heatmaps are extremely popular in fields such as bioinformatics, they remain a severely underutilized visualization tool in modern data analysis. This paper introduces superheat, a new R package that provides an extremely flexible and customizable platform for visualizing complex datasets. Superheat produces attractive and extendable heatmaps to which the user can add a response variable as a scatterplot, model results as boxplots, correlation information as barplots, and more. The goal of this paper is two-fold: (1) to demonstrate the potential of the heatmap as a core visualization method for a range of data types, and (2) to highlight the customizability and ease of implementation of the superheat R package for creating beautiful and extendable heatmaps. The capabilities and fundamental applicability of the superheat package will be explored via three reproducible case studies, each based on publicly available data sources.
BibTeX
@article{barter2015superheat,
  author = {Rebecca Barter and Bin Yu},
  title = {{Superheat: An R Package for Creating Beautiful and Extendable Heatmaps for Visualizing Complex Data}},
  journal = {Journal of Computational and Graphical Statistics},
  year = {2015},
}
Figure