Lifting Heterogeneous Data into Semantic Knowledge Graphs

134 concepts124 papers1 proposal

Data is trapped in structural silos. A relational database stores information in rigid tables; an API returns nested JSON trees; legacy enterprise systems dump data into XML or flat CSV files. Each of these formats speaks a different syntactic and semantic language. When researchers attempt to connect these disparate sources—such as linking chemical compounds to biological pathways, tracking public procurement across corporate registries, or connecting agricultural fields to corporate owners—they hit a wall. Traditional ETL (Extract, Transform, Load) pipelines require writing custom, brittle code for every single pair of data sources. If a schema changes, the pipeline breaks.

The core insight of semantic data transformation is to bypass rigid, localized schemas entirely by "uplifting" heterogeneous data into a single, universal data structure: the directed, edge-labeled graph, formalized as Resource Description Framework (RDF) triples. By decoupling a dataset's underlying meaning from its physical storage format, this approach turns isolated data silos into an interconnected, machine-understandable web of global knowledge.

Anatomy of the Uplift: From Tables and Trees to Triples

To transform a flat spreadsheet or a nested JSON tree into a graph, data must be broken down into atomic facts consisting of a subject, a predicate, and an object. Every entity and property is assigned a global identifier (a URI), ensuring that when two different datasets mention the same concept, they automatically connect.

Tabular Data

In a spreadsheet or relational table, a row typically represents an entity, a column represents a property, and a cell contains the value. As demonstrated by Martin G. Skjæveland et al. in their work on Publishing the Norwegian Petroleum Directorate's FactPages as Semantic Web Data, this tabular structure can be systematically lifted into a semantic graph. A row representing an oil well becomes the subject URI, the column header becomes the predicate URI, and the cell value becomes the object. To ensure data quality, this sequence often utilizes a stepwise enrichment workflow where data is cleaned in a relational environment before applying complex ontological mappings. Similarly, Pei-Hung Lin et al. in Making Machine Learning Datasets and Models FAIR for HPC: A Methodology and Case Study show that mapping local tabular headers to global ontology terms allows fine-grained data elements to become fully interoperable across different high-performance computing platforms.

Hierarchical Data (XML and JSON)

For tree-like structures, the transformation flattens nested relationships into explicit graph edges. In The Yelp Collaborative Knowledge Graph, Theis E. Jendal et al. demonstrate how nested JSON data structures can be unrolled into RDF triples, creating a granular, interconnected representation without losing attribute information or contextual social network data. For XML schemas, Shusaku Egami et al. show in An Ontology-based Approach for Semantic Interoperability in Air Traffic Information Management that automated transformation processes can convert legacy XML data schemas into formal Web Ontology Language (OWL) representations, preserving original domain vocabularies and structural constraints while enabling graph-based integration.

Declarative Mapping: Separating Logic from Execution

Instead of writing hardcoded scripts to perform these transformations, modern semantic pipelines rely on declarative mapping languages, such as RML (RDF Mapping Language). Declarative mapping separates the transformation logic (defining which data fields map to which semantic concepts) from the execution engine.

This strategy separates the transformation logic from the source data format, facilitating scalable and consistent data ingestion. For example, in P2P-O: A Purchase-To-Pay Ontology for Enabling Semantic Invoices, Michael Schulze et al. employ RML to define rules that seamlessly transform heterogeneous XML-based invoices into structured purchase-to-pay knowledge graphs.

Because writing these mappings manually can be tedious, recent work focuses on automation. In Spread2RML: Constructing Knowledge Graphs by Predicting RML Mappings on Messy Spreadsheets, Markus Schröder et al. introduce an automated mapping framework that utilizes a library of predefined templates. By integrating RML references with specific transformation functions from the Function Ontology, their system automatically predicts mapping rules for messy spreadsheets, capturing data hidden in cell formatting (like colors or bold text) and converting diverse data types into structured semantic formats without manual intervention.

Materialization vs. Virtualization

Once mapping rules are established, architects face a fundamental choice in how to expose the resulting semantic data: materialization or virtualization.

Data Sources ---------> [ Mapping Rules ] ---------> Semantic Graph
  (SQL, JSON, XML)             |
                               v
       +-----------------------+-----------------------+
       |                                               |
       v                                               v
[ Materialization ]                             [ Virtualization ]
- Physically generates triples                  - Leaves data in native stores
- Stored in a Triple Store                      - Creates a Virtual Knowledge Graph
- Fast, complex global queries                  - On-the-fly SPARQL-to-SQL translation
- Requires storage & periodic updates           - Real-time data, zero storage overhead

Materialization

This strategy physically executes the transformation rules, generating a static dataset of RDF triples that are loaded into a graph database (a triple store). This is ideal for massive, stable datasets where query performance is paramount. A prime example is detailed by Michael Färber et al. in SemOpenAlex: The Scientific Landscape in 26 Billion RDF Triples, where a semi-automated pipeline converts raw JSON metadata into a standardized 26-billion-triple RDF knowledge graph, enabling complex semantic queries across global scientific output.

Virtualization

Alternatively, data can be left in its original legacy stores while exposing a "Virtual Knowledge Graph" (VKG). When a user executes a semantic query (written in SPARQL), the virtualization engine uses the mapping rules on the fly to translate the graph query into native queries (such as SQL) optimized for the underlying databases.

In Coordinating Social Care and Healthcare using Semantic Web Technologies, Spyros Kotoulas et al. leverage virtual RDF to unify highly heterogeneous data across healthcare and social care silos. This allows legacy systems to be integrated into a broader knowledge network on demand, maintaining their original data structures and storage formats while avoiding the cost and privacy concerns of a centralized data warehouse. To democratize this approach, Haofen Wang et al. developed VKGBuilder in VKGBuilder - A Tool of Building and Exploring Vertical Knowledge Graphs, automating the pipeline from relational schema mapping to conflict resolution so that non-experts can build and explore vertical knowledge graphs.

Why It Works: The Power of the Unified Graph

The ultimate value of semantic data transformation lies in the mathematical elegance of graph unions. In a relational database, joining data across two different systems requires matching foreign keys and creating intermediate tables. If you want to join 25 databases, the complexity explodes.

In an RDF graph, because every entity is identified by a global URI, merging datasets is computationally trivial: it is simply the set-theoretic union of their triples. If Dataset A contains [Compound X] -> [targets] -> [Protein Y] and Dataset B contains [Protein Y] -> [associatedWith] -> [Disease Z], merging the two graphs instantly creates a path from [Compound X] to [Disease Z].

This capability unlocks advanced data mining and discovery:

By transforming data from localized, format-dependent structures into a shared semantic graph, organizations can stop building brittle, point-to-point data integrations and instead contribute to a growing, self-describing corporate or scientific knowledge ecosystem.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_semantic_data_integration_semantic_data__d3b317b4,
  title = {semantic data integration (semantic data transformation)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-data-integration-semantic-data-transformation/rdr_d3b317b42b23457d9a2d70287013da50},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_semantic_data_integration_semantic_data__d3b317b4_v1,
  title = {semantic data integration (semantic data transformation)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-data-integration-semantic-data-transformation/rdr_d3b317b42b23457d9a2d70287013da50/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake