QUALITY-AWARE INTEGRATION AND WAREHOUSING OF GENOMIC DATA

Laure Berti‐Équille, Fouzia Moussouni2014

TL;DR

This paper tackles the critical challenge of integrating and warehousing vast, heterogeneous biomedical and genomic data by proposing a quality-aware architecture. It introduces a set of quality criteria and a modular framework to ensure data reliability, crucial for advancing biomedical research and developing new diagnostics and therapeutics.

What Problem Does This Paper Address?

The explosion of biotechnological platforms has led to an unprecedented surge in biomedical data, from gene sequences and expression levels to clinical records. Researchers worldwide collaborate and share this data, necessitating dynamic integration and warehousing into a single repository. However, this data is inherently complex: it's rich, heterogeneous, distributed across numerous sources, often asynchronously updated, prone to errors and omissions, and constantly evolving. Simply aggregating this data without considering its quality can lead to flawed research, wasted resources, and ultimately, hinder the development of new medical tools. The paper addresses the need for robust methods to manage and ensure the quality of this data during integration and warehousing.

Methods

The authors propose a "quality-aware" approach to data integration and warehousing, building upon their experience with the GEDAW (Gene Expression Data Warehouse) system. Their methodology involves:

  • Data Quality Criteria: Defining a comprehensive set of quality criteria categorized into Bio-Knowledge-based (originality, domain authority), Schema-based (completeness, level of detail, redundancy), and Contextual (freshness, consolidation degree).
  • Entity Resolution and Record Linkage: Employing sophisticated techniques to identify and link equivalent biological entities across different data sources, including exact and near-exact key matching, probabilistic linkage, and analysis of semantic and cognitive information.
  • Biomedical Data Scrubbing and Conflict Resolution: Developing rules for mapping source records and reconciling conflicting values. This includes structural, semantic, and cognitive mapping rules, leveraging tools like BLAST and ontologies (UMLS, GO) for unification.
  • Quality Metadata Management: Storing quality information in XML files linked to each record, using a flexible XML schema to represent criteria, scores, generation dates, and comments. This metadata is generated by pre- and post-validation programs.
  • Modular Architecture: Proposing a functional and modular architecture with various "check-point" modules for data quality control at different stages of the integration and warehousing process, including pre-validation, ETL, and post-validation steps.

Datasets

The paper discusses the use of various public biological databanks as data sources for their GEDAW system. These include:

  • GenBank: For genomic features of genes (in XML format).
  • Public Databanks: Primarily in XML format, containing information on genes expressed in the liver.
  • Biomedical Ontologies and Terminologies: Such as UMLS, MeSH, and GO, also stored as XML documents.
  • DNA Chips Home Experiments: Data from these experiments.
  • Medical Records: Clinical data.

The paper does not introduce new benchmark datasets but rather focuses on integrating and managing existing, real-world biomedical data.

Key Insights and Ideas

  • Data Quality is Paramount, Not an Afterthought: The paper strongly emphasizes that data quality cannot be an afterthought in biomedical data integration. It must be an integral part of the entire process, from source selection to warehousing.
  • Heterogeneity is a Major Hurdle: The sheer diversity of data formats, structures, and semantics across biological databases makes integration a significant challenge.
  • Entity Resolution is Complex but Crucial: Accurately identifying and linking the same biological entity (e.g., a gene) across different sources is non-trivial and requires sophisticated techniques beyond simple keyword matching.
  • A Multi-faceted Approach to Quality: No single quality metric is sufficient. A combination of bio-knowledge, schema, and contextual criteria is needed for a holistic assessment.
  • Metadata is Key to Transparency and Trust: Storing detailed quality metadata alongside the data provides transparency, allows for informed decision-making, and builds trust in the integrated dataset.
  • Modularity Enhances Flexibility: A modular architecture allows for the flexible implementation of quality checks and controls at various points in the data pipeline, adapting to different needs and evolving requirements.

Key Results

The paper presents a conceptual framework and a set of proposed quality criteria rather than quantitative experimental results with specific performance metrics. The key "results" are:

  • A Defined Set of Quality Criteria: The paper provides a structured list of quality criteria relevant to biomedical data, offering a framework for evaluating data quality.
  • A Proposed Architecture: The paper outlines a modular architecture for quality-preventive integration and warehousing, demonstrating how different quality control mechanisms can be integrated.
  • Lessons Learned: The authors share practical insights gained from building the GEDAW data warehouse, highlighting common data quality issues encountered in the biomedical domain.
  • Illustrative Example: The paper uses the HFE gene as an example to demonstrate how different records from GenBank can be integrated and how quality criteria like completeness and relevancy are applied.

Limitations and Future Work

Limitations:

  • Manual Curation Still Necessary: While the paper advocates for automatic and impartial evaluation, it acknowledges that human expert curation is still the current solution for ensuring data quality in biomedical databanks, highlighting its drawbacks (slowness, potential conflicts).
  • Complexity of Formalization: The high level of expertise required for entity resolution and data interpretation makes formalizing these processes challenging.
  • Performance Considerations: The paper notes that integrating massive datasets can burden data warehouses, and their approach aims to minimize this by integrating sources one at a time.

Future Work:

  • Extending GEDAW: The authors plan to continue developing GEDAW to expand its mapping and filtering rules and to implement the described quality check-points more comprehensively.
  • Validating the Approach: The ultimate goal is to validate the proposed quality metrics and functional architecture, demonstrating their usefulness for biomedical researchers.
  • Automating Quality Evaluation: Further research is needed to develop more automatic, impartial, and independent data quality evaluation techniques and tools for structured, semi-structured, and textual biomedical data.

Summary updated

Abstract

In human health and life sciences, researchers extensively collaborate with each other, sharing biomedical and genomic data and their experimental results. This necessitates dynamically integrating different databases or warehousing them into a single repository. Based on our past experience of building a data warehouse called GEDAW (Gene Expression Data Warehouse) that stores data on genes expressed in the liver during iron overload and liver pathologies, and also relevant information from public databanks (mostly in XML format), DNA chips home experiments and medical records, we present the lessons learned, the data quality issues in this context and the current solutions we propose for integrating and warehousing biomedical data. This paper provides a functional and modular architecture for data quality enhancement and awareness in the complex processes of integration and warehousing of biomedical data.
BibTeX
@misc{bertiequille2014quality,
  author = {Laure Berti‐Équille and Fouzia Moussouni},
  title = {{QUALITY-AWARE INTEGRATION AND WAREHOUSING OF GENOMIC DATA}},
  year = {2014},
}
Figure