Are Large Language Models the New Interface for Data Pipelines?

Sylvio Barbon Junior, Paolo Ceravolo, Sven Groppe, Mustafa Jarrar, Samira Maghool, Florence Sèdes, Soror Sahri, Maurice van Keulen

Abstract

This position paper explores the transformative potential of Large Language Models (LLMs) as the primary interface for complex data pipelines. By acting as a "bridge" between users and specialized technologies like Knowledge Graphs, Automated ML, and Explainable AI, LLMs could democratize data science. However, realizing this vision requires overcoming significant hurdles in energy consumption and factual reliability.

Introduction

Building and managing data pipelines—the systems that move, transform, and analyze data—traditionally requires deep technical expertise in SQL, Python, and machine learning. As "Big Data" grows in complexity, these pipelines often become "black boxes" that are difficult for non-experts to navigate or trust. This paper addresses whether LLMs can serve as a universal, natural-language interface for these pipelines, making data systems more intuitive and transparent.

A Conceptual Framework for Integration

Rather than introducing a new algorithm, this work proposes a conceptual framework for "technological synergy." The authors map out how LLMs should interact with four key domains to improve integration across human, computer, and knowledge dimensions.

LLMs as a central interface for data pipelines.

The framework identifies four primary pillars:

  1. Big Data Analytics (BDA): LLMs link raw data to real-world concepts, automating tasks like entity resolution and schema matching.
  2. Knowledge Graphs (KG): KGs provide "factual anchors" to reduce hallucinations, while LLMs simplify querying complex graphs using natural language.
  3. Explainable AI (XAI): LLMs translate complex model decisions into human-readable narratives to improve human integration.
  4. Automated Machine Learning (AutoML): LLMs assist in algorithm selection and hyperparameter optimization based on plain English requirements.

Synergies and Challenges

A critical insight is the bidirectional relationship between LLMs and Knowledge Graphs. While KGs provide structured ground truth, LLMs provide the flexible interface KGs have historically lacked. To address high energy costs—where a single GPT-4 query can consume up to 33 times more energy than a standard Google search—the authors suggest a "generate once" approach. In this model, LLMs generate efficient, traditional code or configurations that run repeatedly at low cost, rather than re-solving the problem for every query.

Current performance metrics highlight a significant reliability gap. For instance, top-tier models like GPT-4 achieve roughly 54.89% accuracy on text-to-SQL tasks, far below the human benchmark of ~93%. Furthermore, research indicates that models trained on human-annotated data still consistently outperform those trained on synthetic labels generated by other LLMs.

Conclusion and Future Outlook

Several hurdles remain before LLMs can become the standard interface for data pipelines. There is an inherent irony in using one "black box" (an LLM) to explain another in XAI, and the carbon footprint of inference remains a primary concern. Ultimately, while LLMs will likely automate "low-level" tasks, the role of the data scientist will evolve toward high-level problem solving, ethical oversight, and interdisciplinary collaboration rather than disappearing entirely.

Summary updated

BibTeX
@inproceedings{junior2024are,
  author = {Sylvio Barbon Junior and Paolo Ceravolo and Sven Groppe and Mustafa Jarrar and Samira Maghool and Florence Sèdes and Soror Sahri and Maurice van Keulen},
  title = {{Are Large Language Models the New Interface for Data Pipelines?}},
  booktitle = {BiDEDE@SIGMOD},
  year = {2024},
  doi = {10.1145/3663741.3664785},
}
Figure