Converting tabular data into images for deep learning with convolutional neural networks

Yitan Zhu, Thomas Brettin, Fangfang Xia, Alexander Partin, Maulik Shukla, Hyunseung Yoo, Yvonne A. Evrard, James H. Doroshow, Rick StevensScientific 2021 · DOI

Abstract

Convolutional neural networks (CNNs) have revolutionized fields like computer vision and speech processing by exploiting spatial and temporal dependencies. However, tabular data—the most common format in many scientific and industrial domains—often lacks an inherent spatial structure, making it difficult to apply CNNs effectively. We present the Image Generator for Tabular Data (IGTD), a novel algorithm that transforms tabular features into compact images by mapping similar features to neighboring pixels. By minimizing the rank-based difference between feature distances and pixel distances, IGTD preserves local feature relationships. Our evaluation on anti-cancer drug response datasets demonstrates that CNNs trained on IGTD-generated images outperform traditional machine learning models and existing table-to-image methods.

Introduction

The success of deep learning, particularly CNNs, is largely attributed to their ability to learn hierarchical representations from data with local correlations. In images, adjacent pixels are often related; in speech, consecutive samples are temporally linked. Tabular data, however, consists of independent columns where the ordering is arbitrary. Standard approaches like Multi-Layer Perceptrons (MLPs) treat all features equally, potentially missing complex local interactions.

Previous attempts to convert tables to images, such as DeepInsight (using t-SNE) or OmicsMapNet (using biological pathways), suffer from sparsity, high computational costs, or a reliance on domain-specific knowledge. IGTD addresses these issues by providing a compact, unsupervised, and computationally efficient framework for feature-to-pixel mapping.

The IGTD Algorithm

The core objective of IGTD is to find an optimal assignment of $N$ features to $N$ pixel positions in a $M \times M$ grid (where $M^2 \approx N$). The algorithm ensures that features that are "close" in the original feature space are also "close" in the resulting image.

Rank-Based Optimization

IGTD utilizes a rank-based strategy to handle varying data scales and outliers. The process involves:

  1. Distance Calculation: Computing pairwise distances between all features (e.g., using Euclidean distance or Pearson correlation) and pairwise distances between all pixel coordinates in the grid.
  2. Ranking: Converting these distances into ranks.
  3. Objective Function: Minimizing the difference between the feature distance ranks and the pixel distance ranks.
  4. Iterative Refinement: Swapping feature assignments to reduce the objective function value until convergence.
Optimization process of IGTD: (a) initial feature distance matrix, (b) target pixel distance matrix, (c) optimized feature distance matrix, and (d) convergence of the objective function.

This approach results in a compact image where every pixel represents a feature, and the spatial arrangement reflects the underlying data structure.

Experimental Evaluation

We evaluated IGTD using two benchmark datasets for anti-cancer drug response prediction: the Cancer Therapeutics Response Portal (CTRP) and the Genomics of Drug Sensitivity in Cancer (GDSC). These datasets include gene expression profiles for cancer cell lines and molecular descriptors for various drugs.

We selected the 2,500 most variable genes and 2,500 most variable drug descriptors, transforming them into $50 \times 50$ pixel images. These images were then used to train CNNs to predict drug sensitivity (measured as Area Under the Curve, or AUC).

Key Results

The results highlight several advantages of the IGTD approach:

  • Performance: CNNs using IGTD representations achieved an average $R^2$ of 0.856 on CTRP and 0.74 on GDSC, significantly outperforming traditional models like LightGBM, Random Forest, and standard DNNs.
  • Efficiency: Unlike t-SNE-based methods that produce sparse images (e.g., $227 \times 387$), IGTD produces dense $50 \times 50$ images. This compactness reduces memory usage and accelerates CNN training.
  • Structural Preservation: Using a Local Heterogeneity (LH) metric, we confirmed that IGTD preserves the neighborhood structure of features better than competing methods like REFINED.
  • Computational Speed: IGTD's optimization is fast; even a single-CPU implementation outperformed parallelized versions of alternative algorithms.

Conclusion and Future Work

IGTD provides a robust and efficient way to bridge the gap between tabular data and the power of convolutional architectures. By transforming features into a spatially meaningful grid, it enables CNNs to capture local interactions that are often lost in traditional tabular models.

Future research will explore extending IGTD to 3D volumes, handling datasets with vastly different feature counts through automated selection or padding, and investigating non-rectangular image shapes to better represent complex data topologies.

Summary updated

Abstract

Convolutional neural networks (CNNs) have been successfully used in many applications where important information about data is embedded in the order of features, such as speech and imaging. However, most tabular data do not assume a spatial relationship between features, and thus are unsuitable for modeling using CNNs. To meet this challenge, we develop a novel algorithm, image generator for tabular data (IGTD), to transform tabular data into images by assigning features to pixel positions so that similar features are close to each other in the image. The algorithm searches for an optimized assignment by minimizing the difference between the ranking of distances between features and the ranking of distances between their assigned pixels in the image. We apply IGTD to transform gene expression profiles of cancer cell lines (CCLs) and molecular descriptors of drugs into their respective image representations. Compared with existing transformation methods, IGTD generates compact image representations with better preservation of feature neighborhood structure. Evaluated on benchmark drug screening datasets, CNNs trained on IGTD image representations of CCLs and drugs exhibit a better performance of predicting anti-cancer drug response than both CNNs trained on alternative image representations and prediction models trained on the original tabular data.
BibTeX
@inproceedings{zhu2021converting,
  author = {Yitan Zhu and Thomas Brettin and Fangfang Xia and Alexander Partin and Maulik Shukla and Hyunseung Yoo and Yvonne A. Evrard and James H. Doroshow and Rick Stevens},
  title = {{Converting tabular data into images for deep learning with convolutional neural networks}},
  booktitle = {Scientific},
  year = {2021},
  doi = {10.1038/s41598-021-90923-y},
}
Figure