Knowledge Graphs for Digital Transformation Monitoring in Social Media

Danilo Dessì, Gianni Fenu, Sergio Consoli, Vanni Zavarella, Francesco Osborne, Davide Buscaldi, Simone Angioni, Diego Reforgiato RecuperoTEXT2KG/DQMLKG@ESWC 2024 · OpenReview

Introduction

While information extraction (IE) from formal texts like scientific papers or patents has matured significantly, social media remains a challenging frontier. Platforms like X (formerly Twitter) are characterized by noisy text, heavy use of hashtags, slang, and a lack of formal grammatical structure. Existing IE pipelines often struggle with scalability or are too rigid for the evolving nature of digital discourse.

To address this, researchers have introduced Triplétoile, an automated pipeline designed to transform chaotic social media posts into structured, high-quality Knowledge Graphs (KGs). The system focuses on Open-Domain Information Extraction—identifying entities and relationships without a predefined list of categories—specifically for monitoring complex trends like "Digital Transformation."

The Triplétoile Architecture

The Triplétoile pipeline consists of three main phases designed to move from raw text to refined semantic triples (subject-relation-object).

The multi-stage architecture of the Triplétoile pipeline, from data preprocessing to knowledge graph generation.
  1. Triple Extraction: Utilizing the spaCy library for dependency parsing, the system identifies nominal phrases as potential entities and connects them via predicative triples (verbs acting as relations). A critical component is coreference resolution, which ensures that pronouns (e.g., "it") are correctly linked to previously mentioned entities.
  2. Entity Refining: Social media handles and hashtags (e.g., #DigitalTransformation) are normalized into human-readable concepts. These are then grounded by linking them to DBpedia, allowing the system to recognize that different surface strings refer to the same real-world entity.
  3. Relation Refining: To manage the high variability of verbs used in social media, the authors employ unsupervised clustering. By using word embeddings (GloVe), reducing dimensionality with UMAP, and clustering with HDBSCAN, the system maps various verbs to a single representative label, effectively generalizing relationships.

Methodology and Data

The study utilized two primary data sources for evaluation:

  • DTSMM Dataset: A collection of approximately 100,000 tweets from 2022 related to the "Digital Transformation" domain.
  • Benchmark Dataset: A manually curated set of 500 triples extracted by the pipeline, evaluated by three domain experts to establish a ground truth for precision.

Key Results and Performance

Triplétoile demonstrates significant improvements over existing open-domain extraction tools in both accuracy and yield. In a human expert assessment, the pipeline achieved an overall accuracy of 96%.

Comparison of extraction methods showing Triplétoile's superior precision and triple yield.

As shown in the comparative analysis, Triplétoile achieved a precision of 0.82, outperforming the next best competitor, the Dependency-based Extractor (0.77). Furthermore, it generated a significantly higher number of valid triples (663) than other high-precision methods, suggesting superior recall and data coverage. The final Knowledge Graph generated for the Digital Transformation domain contains 22,270 triples and 18,693 unique entities.

Discussion and Future Work

A key innovation of Triplétoile is the use of clustering over fixed classification, allowing the Knowledge Graph to grow organically as new tech trends emerge. Additionally, the authors utilized "reification," turning statements into objects with metadata (e.g., frequency of mention), which is essential for monitoring trends over time.

Current limitations include a lack of native entity typing (e.g., categorizing "Microsoft" as a "Company" without external DBpedia links). Future work will focus on integrating Large Language Models (LLMs) like GPT-4 to move toward supervised relation mapping and using these KGs as "knowledge plugins" to provide factual grounding for generative AI.

Summary updated

BibTeX
@inproceedings{dessi2024knowledge,
  author = {Danilo Dessì and Gianni Fenu and Sergio Consoli and Vanni Zavarella and Francesco Osborne and Davide Buscaldi and Simone Angioni and Diego Reforgiato Recupero},
  title = {{Knowledge Graphs for Digital Transformation Monitoring in Social Media}},
  booktitle = {TEXT2KG/DQMLKG@ESWC},
  year = {2024},
  url = {https://openreview.net/forum?id=xm2tRQiXIG},
}
Figure