AutoChart: A Dataset for Chart-to-Text Generation Task

Jiawen Zhu, Jinye Ran, Roy Ka-Wei Lee, Zhi Li, Kenny T. W. Choo

Introduction

While Natural Language Generation (NLG) has advanced significantly in translating structured data into text, the "chart-to-text" task remains a formidable challenge. Existing datasets often suffer from small scales, "noisy" information not present in the visual, or descriptions that merely list data points without providing meaningful analysis. To bridge this gap, researchers have introduced AutoChart, a large-scale dataset designed to train models to generate sophisticated, human-like analytical descriptions. By integrating statistical data with a linguistic framework of "Rhetorical Moves," AutoChart provides over 10,000 charts and 23,000 descriptions that rival human-written summaries in quality and informativeness.

The AutoChart Framework

The core of AutoChart is a novel framework that ensures perfect alignment between visual charts and high-quality analytical text. The generation process is divided into two primary modules.

AutoChart system architecture for generating chart-description pairs.

Data and Trend Generation

The system utilizes real-world statistical data from sources like the World Bank, supplemented by simulated trends using Brownian motion equations. This allows the dataset to cover a wide spectrum of mathematical patterns, including spikes, constant increases, and plateaus, ensuring that models learn to recognize diverse visual behaviors.

Analytical Description Generation

The "Rhetorical Moves" framework is the primary innovation for text generation. Rather than relying on simple templates, the system structures descriptions based on five logical moves:

  1. Overview: Defining the chart's subject.
  2. Configuration: Describing axes and elements.
  3. Interpretation: Identifying trends and specific data points.
  4. Evaluation: Comparing different variables.
  5. Conclusion: Summarizing the significance of the data.

To enhance linguistic variety, the researchers employed paraphrasing algorithms on human-written templates, preventing the resulting models from producing repetitive, robotic outputs.

Dataset Composition

The resulting AutoChart dataset comprises 10,232 charts and 23,543 corresponding analytical descriptions. It includes four common chart types—scatter plots, line charts, and both vertical and horizontal bar charts—with varied styles and colors. Each entry includes rich JSON metadata, such as bounding boxes for legends and exact data coordinates, making the dataset a valuable resource for both computer vision and NLP tasks. Notably, the descriptions are modeled after the IELTS Academic Writing Task 1, a standard for testing human visual-data summarization skills.

Key Results and Evaluation

The quality of AutoChart was validated through both automatic metrics and human judgment.

Human evaluation results comparing AutoChart (red) to human-written descriptions (blue).

In blind human evaluations, AutoChart's descriptions were rated nearly as high as human-written ones across several dimensions. Specifically, AutoChart achieved scores of 4.2 for Naturalness and Quality, compared to human scores of 4.5 and 4.3, respectively. Remarkably, AutoChart outperformed human descriptions in Informativeness for bar charts (scoring 4.4 vs. 4.3).

Automatic metrics further confirmed these findings, with AutoChart achieving significantly higher BLEU (41.28) and ROUGE (46.11) scores than baseline template methods. Benchmarking existing state-of-the-art models on this dataset revealed BLEU scores in the 10-33 range, highlighting that AutoChart serves as a challenging and necessary benchmark for the field.

Conclusion and Future Directions

AutoChart represents a significant step toward automated, high-quality reporting and improved accessibility for the visually impaired. While the current dataset focuses on four basic chart types, future work aims to expand to more complex visualizations like pie charts and stacked area charts. Additionally, the researchers plan to explore deeper linguistic reasoning to further enhance the logic and fluency of generated summaries, while also addressing the challenge of "noisy" real-world data that lacks the clean formatting of synthetic generation.

Summary updated

Abstract

The analytical description of charts is an exciting and important research area with many applications in academia and industry. Yet, this challenging task has received limited attention from the computational linguistics research community. This paper proposes \textsf{AutoChart}, a large dataset for the analytical description of charts, which aims to encourage more research into this important area. Specifically, we offer a novel framework that generates the charts and their analytical description automatically. We conducted extensive human and machine evaluations on the generated charts and descriptions and demonstrate that the generated texts are informative, coherent, and relevant to the corresponding charts.
BibTeX
@inproceedings{zhu2021autochart,
  author = {Jiawen Zhu and Jinye Ran and Roy Ka-Wei Lee and Zhi Li and Kenny T. W. Choo},
  title = {{AutoChart: A Dataset for Chart-to-Text Generation Task}},
  booktitle = {RANLP},
  year = {2021},
  url = {https://openreview.net/forum?id=xT1N1bXcGIF},
}
Figure