Research on Data Sharing and Integration of Multi-source Information Systems

Gang Xu, Peng Xu, Z. Wang2015 · DOI

Abstract

In the modern digital landscape, information systems often operate as isolated "islands," hindering efficient data sharing and cross-departmental analysis. This paper proposes a robust data sharing and integration architecture designed to bridge these gaps. The architecture is composed of four distinct layers: the Heterogeneous Data Source Layer, the Meta-database Layer, the Data Access Layer, and the Application Layer. Central to this design is the use of the Dublin Core (DC) metadata standard and XML protocols to create a unified data representation. This paper details the design philosophy, the specific metadata elements used, and the step-by-step process for achieving seamless data integration across diverse platforms.

The Challenge of "Information Islands"

As organizations grow, they inevitably adopt a variety of database management systems (DBMS) such as Oracle, MySQL, SQL Server, and Kingbase. Furthermore, data is often stored in disparate formats, ranging from structured tables to unstructured text. This heterogeneity creates "Data Islands," where valuable information is locked within specific systems, inaccessible to others. The lack of a unified standard prevents deep data analysis and limits the "value-added utilization" of an organization's information assets. To solve this, a standardized "translator" is needed to mediate between raw data sources and end-user applications.

A Four-Layer Architectural Framework

The proposed solution is a hierarchical architecture that decouples data storage from data application, allowing for flexible and scalable integration.

The four-layer architecture for data integration.
  1. Heterogeneous Data Source Layer: This foundational layer encompasses all raw data repositories, including various relational databases and unstructured document files.
  2. Meta-database Layer: Acting as the system's "brain," this layer uses XML and the Dublin Core standard to describe underlying data. By focusing on metadata rather than the raw data itself, the system creates a lightweight, searchable index of all available resources.
  3. Data Access Layer: This layer provides dynamic adaptation. It automatically selects the most appropriate connection protocol—such as JDBC, ODBC, JMS, or SOCKET—based on the specific requirements of the requesting application.
  4. Application Layer: The top-most layer where various end-user systems interact with the integrated data environment to fulfill specific business needs.

Metadata Standardization via Dublin Core

A critical insight of this research is the use of the Dublin Core (DC) element set as the universal language for the meta-database. While DC was originally intended for web resources, its 15 basic elements (including Title, Creator, Subject, and Resource Identifier) are remarkably effective at describing "Document-Like Objects" (DLOs) across different domains. By mapping diverse data sources to these 15 standardized elements, the system achieves a "less is more" approach to data organization, ensuring that even complex datasets can be categorized and retrieved efficiently.

The Data Sharing and Integration Process

The integration workflow is designed to be automated and responsive to user needs. The process follows a logical cycle from requirement to fulfillment.

The workflow process for data sharing and integration.

As illustrated in the workflow, the system begins by translating a User Requirement into specific Data Requirements. It then queries the Meta-database to locate the relevant information. Once the data is identified, the system performs an Automatic Match of Data Access Methods to establish a connection to the source. Finally, the data is Applied by the Information System to meet the original user demand, completing the cycle.

Conclusion and Future Work

This four-layer architecture provides a theoretically sound blueprint for overcoming the technical hurdles of multi-source data integration. By leveraging XML-based metadata and standardized access protocols, it allows organizations to treat their disparate data sources as a single, unified resource.

However, it is important to note that this research is currently a formal representation. Future work should focus on empirical implementation and performance testing. Key areas for further study include the system's performance in high-concurrency environments and its ability to handle massive, real-time unstructured datasets. Realizing this architecture in a production environment will be the next step toward truly eliminating information silos.

Summary updated

Abstract

A data sharing and integration architecture of multi-source information systems is designed and this architecture is composed of 4 layers. A metadata format, DUBLIN CORE, is introduced in detail and the design idea is introduced as well. Besides, the process of data sharing and integration is introduced in detail.
BibTeX
@misc{xu2015research,
  author = {Gang Xu and Peng Xu and Z. Wang},
  title = {{Research on Data Sharing and Integration of Multi-source Information Systems}},
  year = {2015},
  doi = {10.2991/ssemse-15.2015.330},
}
Figure