AutoChart: A Dataset for Chart-to-Text Generation Task

Jiawen Zhu, Jinye Ran, Roy Ka-Wei Lee, Kenny T. W. Choo, Zhi Li

Abstract

The analytical description of charts is an exciting and important research area with many applications in academia and industry. Yet, this challenging task has received limited attention from the computational linguistics research community. This paper proposes AutoChart, a large dataset for the analytical description of charts, which aims to encourage more research into this important area. Specifically, we offer a novel framework that generates the charts and their analytical description automatically. We conducted extensive human and machine evaluations on the generated charts and descriptions and demonstrate that the generated texts are informative, coherent, and relevant to the corresponding charts.

Introduction

While Natural Language Generation (NLG) has advanced significantly in domains like conversational agents and news summarization, the task of chart-to-text generation remains a substantial challenge. Automated, high-quality analytical descriptions are essential for making visual data accessible and interpretable. Existing datasets often suffer from scarcity, high annotation costs, or "noise"—where human writers include external context not present in the visual data itself, confusing models trying to learn direct data-to-text mapping.

To address these issues, we introduce AutoChart, a massive, structured dataset designed to mimic the rigor of the IELTS Academic Writing Task 1 (AWT1). By generating data and descriptions synthetically, we provide a "gold standard" for pure data-to-text mapping tasks.

Methodology

The AutoChart framework consists of two primary modules: a Chart Generation Module and an Analytical Description Generation Module.

The AutoChart framework architecture, illustrating the automated pipeline from statistical data to chart-description pairs.

Chart Generation

To ensure a wide variety of mathematical behaviors, the researchers employed a Trend Generation Strategy using Brownian motion equations. This allowed for the creation of specific patterns such as spikes, constant increases, and plateaus. These trends were rendered into four common formats: line charts, scatter plots, and both vertical and horizontal bar charts using the Matplotlib library.

Analytical Description Generation

To produce coherent and diverse text, the system utilizes a Rhetorical Move Analysis based on Swales’ framework of genre analysis. Descriptions are structured into five logical "moves": Overview, Configuration, Interpretation, Evaluation, and Conclusion. Base templates were authored by linguistics researchers and expanded using AI paraphrasing tools to ensure linguistic variety. A synthesis engine then dynamically combines these moves into a narrative based on the specific chart's metadata.

Dataset Characteristics

The resulting AutoChart dataset comprises 10,232 charts and 23,543 corresponding analytical descriptions. Unlike previous datasets like FigureQA or PlotQA, which focus on short-form Q&A, AutoChart provides long-form summaries averaging 140 words and 8 sentences. The use of synthetic trends ensures that models learn to recognize data shapes rather than memorizing historical values, while the metadata-driven generation eliminates background knowledge noise.

Results and Evaluation

We evaluated AutoChart using both human judgment and automatic metrics. In a blind study, human participants assessed the generated descriptions against human-written benchmarks.

Human evaluation results comparing AutoChart descriptions (red) to human-written benchmarks (blue) across three key metrics.

As shown in the evaluation results, AutoChart descriptions (red) were rated nearly as high as human-written ones in naturalness and quality, and actually surpassed them in informativeness. Furthermore, AutoChart significantly outperformed template-based baselines in automatic metrics, achieving a BLEU score of 41.28 compared to the baseline's 33.46. Testing state-of-the-art models on this dataset revealed they still struggle to reach human-level performance, confirming AutoChart as a challenging benchmark.

Conclusion

AutoChart provides a robust benchmark for the chart-to-text generation task. While currently limited to four chart types, the framework demonstrates that combining synthetic trend generation with structured rhetorical moves can produce high-quality, human-like analytical descriptions. Future work will focus on expanding chart diversity and improving the logical reasoning and fluency of the generated text.

Summary updated

Abstract

The analytical description of charts is an exciting and important research area with many applications in academia and industry. Yet, this challenging task has received limited attention from the computational linguistics research community. This paper proposes \textsf{AutoChart}, a large dataset for the analytical description of charts, which aims to encourage more research into this important area. Specifically, we offer a novel framework that generates the charts and their analytical description automatically. We conducted extensive human and machine evaluations on the generated charts and descriptions and demonstrate that the generated texts are informative, coherent, and relevant to the corresponding charts.
BibTeX
@inproceedings{zhu2021autochart,
  author = {Jiawen Zhu and Jinye Ran and Roy Ka-Wei Lee and Kenny T. W. Choo and Zhi Li},
  title = {{AutoChart: A Dataset for Chart-to-Text Generation Task}},
  booktitle = {RANLP},
  year = {2021},
  url = {https://openreview.net/forum?id=xT1N1bXcGIF},
}
Figure