Scalable Analysis of Multi-Modal Biomedical Data

Jaclyn Smith, Yao Shi, Michael Benedikt, Miloš NikolićbioRxiv 2020 · DOI

TL;DR

This paper introduces TraNCE, a novel framework that tackles the challenge of analyzing complex, multi-modal biomedical data at scale. By employing query compilation and a "shredded" data representation, TraNCE significantly improves the efficiency and scalability of integrating diverse datasets, outperforming traditional "flattening" methods.

What Problem Does This Paper Address?

Modern biomedical research relies heavily on integrating vast and diverse datasets, such as genomic sequences, medical images, and clinical records, to understand diseases and develop targeted treatments. These datasets often have complex, nested structures (e.g., JSON-like formats with embedded lists and objects). While distributed processing frameworks like Apache Spark are powerful, they struggle to efficiently handle these complex, nested data types at scale. Traditional approaches often involve "flattening" these structures into simpler, tabular formats, which can lead to data loss, incorrect results, and significant performance bottlenecks, especially when dealing with large inner collections or a small number of top-level records. This gap hinders the ability to perform comprehensive, multi-modal analyses crucial for advancing personalized medicine.

Methods

TraNCE addresses this problem through two key innovations:

  • Query Compilation: TraNCE uses a high-level, declarative language (a variant of nested relational calculus) to describe data analyses. This language allows users to express complex operations on nested data without needing to manage the underlying distributed computing details. The framework then compiles these declarative programs into efficient, executable Spark applications.
  • Shredded Data Representation: Instead of flattening nested data, TraNCE employs a "shredded" representation. This approach transforms nested collections into a set of flat relations, where nested attributes are replaced by "labels" that act as foreign keys. This succinct representation allows for more localized operations and reduces data shuffling, significantly improving performance, especially for deeply nested or skewed data. TraNCE also incorporates skew-resilience mechanisms to handle data imbalances.

Datasets

The paper evaluates TraNCE using several publicly available biomedical datasets, including:

  • 1000 Genomes Project: Used for gene and pathway burden analyses, providing a dataset of genetic variation across a large cohort.
  • The Cancer Genome Atlas (TCGA): Utilized for multi-omics cancer driver gene analysis and clinical exploratory queries, offering a comprehensive collection of cancer genomic and clinical data.
  • STRING database: Used for protein-protein interaction networks.
  • Gene Expression Omnibus (GEO): Used for gene expression data.

The paper does not introduce new benchmark datasets but rather demonstrates TraNCE's performance on existing, large-scale biomedical data sources.

Key Insights and Ideas

  • Nested Data is a Bottleneck: The paper clearly demonstrates that traditional "flattening" approaches, while intuitive, are fundamentally ill-suited for scaling complex, nested biomedical data analysis.
  • Shredding for Scalability: The "shredded" data representation is a crucial innovation. By breaking down nested structures into labeled flat relations, TraNCE avoids the performance penalties of full flattening and enables more efficient distributed processing.
  • Declarative Programming for Usability: The high-level, declarative language simplifies the process of defining complex multi-modal analyses, abstracting away the complexities of distributed systems.
  • Query Compilation Optimizes Performance: The compilation of declarative queries into optimized Spark code, combined with shredding, leads to significant performance gains, especially for complex, multi-stage analyses.
  • Clinical Applications are Feasible: The paper shows that TraNCE can be applied to real-world clinical scenarios, enabling interactive exploration of integrated genomic and clinical data.

Key Results

  • Significant Performance Gains: TraNCE, particularly with its shredded compilation route, consistently outperforms standard flattening methods. For instance, in the driver gene analysis, TraNCE achieved a 7x performance gain over standard methods on the breast cancer dataset and was able to complete analyses that standard methods failed on for the pancancer dataset.
  • Scalability with Data Size: The shredded approach scales much better with increasing numbers of variants compared to standard methods, as illustrated in the gene and pathway burden analyses.
  • Efficiency in Clinical Queries: For clinical exploration programs, TraNCE's shredded compilation was up to 80x faster than standard methods for simple restructuring tasks and still offered significant advantages (up to 9x) for more complex join and aggregation operations.
  • Successful Multi-omics Integration: The framework successfully integrated multiple data types (mutations, copy number, protein-protein interactions, gene expression) to identify potential cancer driver genes.
  • Improved Classification Accuracy: In a tumor origin classification task, the one-vs-rest method using TraNCE-generated features achieved a combined accuracy of 78.44%, outperforming a simple multi-class classifier.

Limitations and Future Work

  • Framework is Experimental: The authors acknowledge that TraNCE is an experimental framework and its development is ongoing.
  • Interface with Learning Algorithms: Future work should focus on enhancing the interface between learning analyses and data integration tasks, potentially by extending the language to support iteration and user-defined functions.
  • Iterative Exploration: The paper suggests exploring scenarios with iterative exploration and aggregation, applicable to both research and clinical applications.
  • Backend Adjustments for Clinical Systems: For clinical systems, future work could involve persisting shredded representations in the backend while presenting integrated data to the user, enabling efficient localized operations for report generation.
  • Consortium and Biobank Data: Consortiums and biobanks could benefit from adopting shredded representations in their backends to optimize data analysis pipelines.

Summary updated

Abstract

Targeted diagnosis and treatment options are dependent on insights drawn from multi-modal analysis of large-scale biomedical datasets. Advances in genomics sequencing, image processing, and medical data management have supported data collection and management within medical institutions. These efforts have produced large-scale datasets and have enabled integrative analyses that provide a more thorough look of the impact of a disease on the underlying system. The integration of large-scale biomedical data commonly involves several complex data transformation steps, such as combining datasets to build feature vectors for learning analysis. Thus, scalable data integration solutions play a key role in the future of targeted medicine. Though large-scale data processing frameworks have shown promising performance for many domains, they fail to support scalable processing of complex datatypes. To address these issues and achieve scalable processing of multi-modal biomedical data, we present TraNCE, a framework that automates the difficulties of designing distributed analyses with complex biomedical data types. We outline research and clinical applications for the platform, including data integration support for building feature sets for classification. We show that the system is capable of outperforming the common alternative, based on “flattening” complex data structures, and runs efficiently when alternative approaches are unable to perform at all. Key Points Modern biomedical analyses are integrated pipelines of data access mechanisms and analysis components that operate on and produce datasets in a variety of complex, domain specific formats. Scalable data integration and aggregation solutions that support joint inference on such large-scale datasets play a key role advancing biomedical analysis. Query compilation techniques that optimize nested data processing are essential for scaling multi-modal, biomedical analysis.
BibTeX
@misc{smith2020scalable,
  author = {Jaclyn Smith and Yao Shi and Michael Benedikt and Miloš Nikolić},
  title = {{Scalable Analysis of Multi-Modal Biomedical Data}},
  year = {2020},
  doi = {10.1101/2020.12.14.422781},
  howpublished = {bioRxiv},
}

Loading related content...

Figure