IBM UFO Repository

Michael N. Gubanov, Lucian Popa, C. T. Howard Ho, Hamid Pirahesh, Jeng-Yih Chang, Shr-Chang ChenPVLDB 2009 · OpenReview

TL;DR

This paper introduces the IBM UFO Repository, a novel object-oriented data integration system that tackles the challenge of unifying disparate data sources. By introducing "Unified Famous Objects" (UFOs) as a common abstraction, the system simplifies data access and management, enabling users to query integrated data without needing to understand the underlying source schemas.

What Problem Does This Paper Address?

The paper addresses the pervasive "Data Integration Curse," where data from various sources (web 2.0 mashups, enterprise databases, biological repositories) is often structured, named, and referenced differently. This structural mismatch makes it incredibly difficult and time-consuming to access, combine, and manage this data, especially for complex domains like biology or large enterprises. Existing solutions often require significant manual effort and are not truly unsupervised. The core problem is the lack of a unified, high-level abstraction that can hide the low-level representation differences and provide a standardized way to interact with diverse data.

Methods

The paper proposes an Object-Oriented Data Integration approach centered around the Unified Famous Object (UFO). UFOs act as a common abstraction layer, similar to how objects in object-oriented programming hide implementation details. The system's architecture includes:

  • UFO Repository: Stores a large collection of UFOs, representing common data objects.
  • Object Discovery and Extraction: The system automatically detects and maps familiar objects within incoming data feeds to existing UFOs. This process is aided by the UFO Factory, a tool built on top of Clio, which provides a GUI for semi-automatic object discovery.
  • Standardized Query Interface: Users interact with the system through UFOs, using a standardized query language (demonstrated with XQuery). This interface is oblivious to the original data source schemas.
  • Data Conversion: Incoming data (e.g., XML, JSON) is parsed, traversed, and flattened to extract objects, which are then unified with the best-matching UFO.

The key novelty lies in the introduction of UFOs as a unifying abstraction and the automated discovery and mapping process, which aims to reduce manual effort.

Datasets

The paper demonstrates the system's capabilities using several real-world data sources:

  • Online Stores: eBay, Yahoo!, NewEgg (for product offers).
  • Biological/Medical Data: NCBI, AD&FTD, SwissProt (for genetic and protein information).

The paper does not introduce new benchmarks but showcases the system's ability to integrate and query data from these existing, diverse sources.

Key Insights and Ideas

  • UFOs as a Universal Abstraction: The core idea is that by defining a set of "Unified Famous Objects," we can create a common language for data integration, abstracting away the complexities of individual source schemas.
  • Automated Discovery is Key: The success of this approach hinges on the ability to automatically discover and map objects from new data sources to existing UFOs, significantly reducing manual effort compared to traditional ETL or data warehousing.
  • Modularity for Data Management: Similar to how object-oriented programming brought modularity to software development, UFOs bring modularity to data integration, making it easier to manage and access complex, heterogeneous data.
  • Querying Without Source Knowledge: The system allows users to query integrated data using the UFO interface without needing to know the specifics of how that data is stored in its original source.

Key Results

The paper demonstrates the effectiveness of the UFO Repository through two query examples:

  • Product Search: A single XQuery against a merged dataset from eBay, Yahoo!, and NewEgg, using the U-Item UFO, automatically returns the best offers for a 32" TV.
  • Disease Diagnosis: Another XQuery, using the U-Sequence UFO, queries mutation records from NCBI, AD&FTD, and SwissProt to automatically perform a disease diagnosis based on a patient's genetic profile.

These examples highlight the system's ability to conceal source-specific details and provide a unified view for complex queries.

Limitations and Future Work

  • Current UFO Model: The repository currently uses a "flat" model of UFOs, and the authors acknowledge a clear need for subclasses and polymorphism (e.g., a U-Truck being a subclass of U-Car).
  • UFO Reuse: While the repository offers UFO collections split by domain, the authors are considering ways to reuse UFOs across domains by leveraging polymorphism.
  • Semi-Automatic Discovery: The UFO Factory offers semi-automatic object discovery, implying that some human intervention might still be required in certain complex scenarios.
  • Future Applications: The paper mentions two main applications: object-oriented data integration and object-oriented data exchange, with the latter being an area of ongoing work. They envision a future where retrieving and accessing data from multiple sources is significantly alleviated.

Summary updated

BibTeX
@article{gubanov2009ibm,
  author = {Michael N. Gubanov and Lucian Popa and C. T. Howard Ho and Hamid Pirahesh and Jeng-Yih Chang and Shr-Chang Chen},
  title = {{IBM UFO Repository}},
  journal = {Proc. VLDB Endow.},
  year = {2009},
  url = {https://openreview.net/forum?id=nR-dfJV7pz},
}
Figure