Abstract
Generative AI (GenAI) has witnessed remarkable progress in recent years, demonstrating impressive performance in various generation tasks across domains like computer vision and computational design. This paper surveys the integration of GenAI into visualization frameworks (GenAI4VIS), leveraging superior generative capacity for diverse operations. We categorize GenAI methods—including sequence, tabular, spatial, and graph generation—across four major visualization stages: data enhancement, visual mapping, stylization, and interaction. By summarizing current algorithms, applications, and limitations, this survey provides insights into the challenges and research opportunities in evaluation, datasets, and the gap between end-to-end models and generative algorithms.
Introduction: A New Frontier for Data Storytelling
Traditional data visualization has long been a "high-floor" activity, requiring mastery of complex grammars like Vega-Lite or D3.js. While automated visualization has been a research goal for decades, early systems often struggled with context and messy, real-world data. The emergence of Generative AI (GenAI) offers a paradigm shift, moving beyond rigid, rule-based software toward flexible, context-aware systems.
However, applying GenAI to visualization introduces a unique challenge: data integrity. Unlike artistic image generation, a visualization must be a faithful representation of underlying numbers. This paper explores how to leverage GenAI's creative power without sacrificing truth, mapping the current landscape of GenAI4VIS.
The GenAI4VIS Framework
The authors organize the GenAI4VIS landscape by mapping generative methods to specific stages of the visualization pipeline.
Categorization by Data Structure and Method
The framework identifies four primary types of generative methods:
- Sequence Generation: Employs RNNs and Transformers to generate visualization code or interaction sequences.
- Tabular Generation: Uses GANs to create synthetic data for privacy protection or data augmentation.
- Spatial Generation: Leverages VAEs and Diffusion models for 2D image synthesis, style transfer, and 3D volumetric rendering.
- Graph Generation: Utilizes Graph Neural Networks (GNNs) to optimize layouts and infer relationships in complex networks.
Mapping Tasks to Generative Methods
The relationship between visualization tasks and these methods is complex. The following diagram illustrates how specific data types are handled by different generative approaches to fulfill core visualization tasks.
Key Pipeline Stages
The survey breaks down the visualization process into four stages where GenAI is currently active:
- Data Enhancement: This involves inferring missing values or embedding metadata into images via steganography.
- Visual Mapping: Translating natural language queries into functional code (NL2VIS), effectively bridging the gap between human intent and technical execution.
- Stylization: Transferring aesthetic styles from artwork to charts or generating "data-driven" infographics that maintain visual appeal without losing data accuracy.
- Interaction: Enabling "Chart Question Answering" (CQA), where users can interact with their data through natural language dialogue.
Key Insights and Results
- The Hybrid Approach: Most successful GenAI4VIS tools are currently "hybrid." They use GenAI for "fuzzy" tasks—like understanding intent—but rely on traditional rule-based engines for rendering to ensure data points remain accurate.
- LLMs as Orchestrators: Systems like LIDA demonstrate that LLMs can act as end-to-end agents, summarizing data, generating goals, and writing code in a single workflow.
- Performance Gains: Modern Transformer-based models (e.g., ncNet) significantly outperform older RNN-based approaches in NL2VIS tasks. Furthermore, Vision-Language Models (VLMs) are moving toward "OCR-free" understanding, perceiving charts holistically rather than just reading text.
Challenges and Future Directions
Despite rapid progress, several "growing pains" remain:
- The Evaluation Gap: Standard AI metrics (like image similarity) are insufficient for visualization. New metrics are needed to measure "data faithfulness" and "analytic utility."
- Hallucination: LLMs can confidently describe non-existent trends. Solving this "truthfulness" problem is critical for high-stakes applications in finance or medicine.
- Dataset Diversity: Current training sets often focus on simple bar and line charts. There is a need for authentic datasets featuring complex, multi-view dashboards.
- Explainability: There is a tension between "black-box" image generation and "explainable" code generation. The field currently favors the latter to maintain user trust and allow for manual refinement.
Summary updated