Figure Captioning with Relation Maps for Reasoning
Lists
Only you see this
Abstract
Figures, such as line plots, pie charts, bar charts, are widely used to convey important information in a concise format. In this work, we investigate the problem of figure caption generation where the goal is to automatically generate a natural language description for a given figure. While natural image captioning has been studied extensively, figure captioning has received relatively little attention and remains a challenging problem. A successful solution to this task has many potential applications, such as: 1) automatic parsing large amount of figures in PDF document; 2) improving user experience by allowing figure content to be accessible to those with visual impairment. To solve this problem, we introduce a dataset FigCAP and propose novel attention mechanism. In order to solve the exposure bias issue, we further train the captioning model with sequence-level policy based on reinforcement learning, which directly optimizes evaluation metrics. Extensive experiments show that the proposed method outperforms the baselines, thus demonstrating a significant potential for automatic generating captions for figures.
BibTeX
@inproceedings{chen2020figure,
author = {Charles Chen and Ruiyi Zhang and Eunyee Koh and Sungchul Kim and Scott Cohen and Ryan A. Rossi},
title = {{Figure Captioning with Relation Maps for Reasoning}},
booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
year = {2020},
doi = {10.1109/wacv45572.2020.9093592},
}Loading related content...