Question-guided Insights Generation for Automated Exploratory Data Analysis

Abhijit Manatkar, Ashlesha Akella, Krishnasuri Narayanam, Sameep MehtaAAAI 2025 · DOI

TL;DR

This paper introduces QUIS, an automated system for Exploratory Data Analysis (EDA) that generates and uses questions to uncover insights in tabular data. By combining a question generation module (QUGEN) with an insight extraction module (ISGEN) and employing a beam-search strategy, QUIS aims to make EDA more efficient and accessible, even without extensive user domain knowledge.

What Problem Does This Paper Address?

Exploratory Data Analysis (EDA) is crucial for understanding complex datasets, but it often requires significant domain expertise and proficiency in data exploration techniques. Existing automated EDA systems can be limited by predefined goals or require extensive dataset-specific training, making them challenging to use. This paper addresses the need for a fully automated EDA system that can uncover meaningful insights by intelligently generating and utilizing questions, thereby reducing the reliance on user expertise and computational resources.

Methods

QUIS is built around two core modules:

  • QUGen (Question Generation): This module leverages a large language model (LLM), specifically Llama-3-70b-instruct, to generate "insight cards." These cards contain a reason for the insight, a semantically relevant question, and a breakdown column (B) and measure (M) pair. The generation process is inspired by chain-of-thought prompting and iteratively refined using previously generated cards as in-context examples.
  • ISGen (Insight Generation): This module takes the questions generated by QUGen and uses them to explore the dataset. It employs a beam-search heuristic to efficiently search through subspaces (subsets of data defined by filters) to find statistically significant insights. The search is guided by an "interestingness score" to prune less valuable insights. QUIS supports six predefined insight types, including trends, outliers, and change points.

A key aspect of ISGen is its "dig-deeper" feature, which allows users to select subspaces for more in-depth analysis. To maintain insight quality during this deeper exploration, QUIS prioritizes identifying insights with diverse types.

Datasets

The paper does not explicitly list specific datasets used for training or evaluation. However, it references the "CarSales" dataset (Bhatia 2017) as an example for illustrating SQL queries and the concept of a breakdown column and measure.

Key Insights and Ideas

  • Question-Driven EDA: The core innovation is using generated questions as the primary driver for data exploration, making the process more intuitive and guided.
  • LLM for Insight Card Generation: Leveraging LLMs for generating structured "insight cards" (reason, question, B, M) streamlines the initial phase of insight discovery.
  • Beam Search for Subspace Exploration: Employing a beam-search heuristic is crucial for efficiently navigating the vast space of possible data subspaces without exhaustive search.
  • Interestingness Score for Pruning: An interestingness score helps filter out less significant insights, improving the quality and relevance of the results presented to the user.
  • "Dig-Deeper" for Granular Analysis: The ability for users to select specific insights and explore them further in more refined subspaces offers a powerful interactive capability.
  • Insight Type Heterogeneity Prioritization: This strategy is vital for preventing redundant insights during deeper exploration and ensuring a diverse set of findings.

Key Results

The paper focuses on the methodology and system design of QUIS. While specific quantitative results (e.g., accuracy metrics, speedup percentages) are not detailed in this excerpt, the authors claim that QUIS can uncover statistically significant insights by leveraging data statistics and table semantics. The system allows users to control parameters like beam width, beam depth, and expansion factor to manage the quality and quantity of generated questions and insights. The technical details of the system demonstration are referred to in a separate publication (Manatkar et al. 2024).

Limitations and Future Work

  • LLM Dependence: The reliance on LLMs for question generation might introduce biases or limitations inherent to the LLM's training data and capabilities.
  • Parameter Tuning: While QUIS offers knobs for customization, finding the optimal parameter settings for different datasets might still require some experimentation.
  • Dataset Specificity: The paper mentions that some existing Auto EDA approaches require dataset-specific training, implying that QUIS's ability to generalize across diverse datasets without extensive fine-tuning is a key advantage, but its performance on extremely varied or complex datasets might warrant further investigation.
  • Future Work: The paper implicitly suggests future work in refining the LLM prompting strategies, exploring more sophisticated subspace search algorithms, and potentially incorporating more advanced insight types. The "dig-deeper" feature also opens avenues for more interactive and user-guided exploration workflows.

Summary updated

BibTeX
@inproceedings{manatkar2025question,
  author = {Abhijit Manatkar and Ashlesha Akella and Krishnasuri Narayanam and Sameep Mehta},
  title = {{Question-guided Insights Generation for Automated Exploratory Data Analysis}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i28.35360},
}
Figure