The DIDI dataset: Digital Ink Diagram data

Philippe Gervais, Thomas Deselaers, Emre Aksan, Otmar HilligesarXiv 2020 · arXiv

Abstract

The DIDI dataset introduces over 58,000 hand-drawn digital ink diagrams, providing a rich resource for moving beyond static OCR toward intelligent, interactive diagramming tools. Unlike traditional datasets, DIDI captures the dynamic stroke-by-stroke creation process, including sequence, timing, and pressure. By linking text labels and geometric shapes semantically within complex flowcharts, the dataset enables research into joint modeling of handwriting and sketching.

Introduction

Digital ink—data captured via styluses and touchscreens—offers a significantly richer representation of human input than static images. It records the temporal sequence and physical dynamics of strokes, which are essential for interactive editing and automated "beautification" of sketches. However, progress in digital ink recognition has been hindered by the lack of large-scale, complex datasets. Most existing resources are limited to isolated characters or simple shapes, failing to represent the semantic complexity of real-world diagrams like flowcharts.

The DIDI Dataset

The DIDI dataset addresses this gap by providing approximately 58,655 total drawings. This corpus is orders of magnitude larger than previous benchmarks, such as the Czech Technical University dataset (972 diagrams). The data is categorized into two main subsets: 22,287 diagrams containing text labels and 36,368 diagrams consisting only of shapes.

A collage showing the diversity of hand-drawn diagrams and layouts within the DIDI dataset.

Methodology

The researchers employed a "prompted data collection" strategy to ensure both variety and high-quality ground truth.

  1. Diagram Generation: Using GraphViz, thousands of unique flowchart "dot" files were programmatically generated. These varied in complexity (2 to 6 nodes) and utilized diverse shapes (boxes, ovals, diamonds, octagons) and thematic text labels.
  2. Data Collection: Participants used an Android application on tablets and Chromebooks to trace or draw over these generated prompts.
  3. Two-Phase Collection: To support various research objectives, data was collected in two sessions—one including text labels and one with empty nodes.
A simple example of a diagram prompt used during data collection, showing a 'Reload' to 'Move' transition.

Data Format and Characteristics

The dataset is released in NDJSON format. Each entry includes a unique key, the original prompt ID, and the drawing data represented as an array of strokes. Each stroke contains precise x/y coordinates and millisecond-accurate timestamps, allowing for the reconstruction of the drawing process.

Metric Value
Total Drawings 58,655
Unique Participants 364
Unique Diagram Layouts 6,555
Median Strokes per Drawing 14
Median Points per Drawing 1,559

Key Insights

  • Temporal Dynamics: The inclusion of timestamps and stroke order allows models to learn the intent and process of drawing, which is vital for real-time assistive tools.
  • Semantic Co-existence: DIDI emphasizes that text and shapes are intrinsically linked. For instance, a diamond shape in a flowchart typically denotes a decision point, and the text within it provides the specific logic.
  • Synthetic-to-Real Bridge: By basing prompts on GraphViz-generated structures, the authors provide perfect ground truth for the underlying graph topology, a task that is notoriously difficult with purely hand-labeled data.

Conclusion

The DIDI dataset represents a significant leap forward in scale and complexity for digital ink research. While currently focused on flowcharts, it lays the groundwork for "interactive graphical symbolic understanding." Future work may expand this methodology to other domains, such as architectural or circuit diagrams, ultimately enabling tools that can seamlessly convert messy whiteboard sketches into professional digital assets.

Summary updated

Abstract

We are releasing a dataset of diagram drawings with dynamic drawing information. The dataset aims to foster research in interactive graphical symbolic understanding. The dataset was obtained using a prompted data collection effort.
BibTeX
@misc{gervais2020didi,
  author = {Philippe Gervais and Thomas Deselaers and Emre Aksan and Otmar Hilliges},
  title = {{The DIDI dataset: Digital Ink Diagram data}},
  year = {2020},
  eprint = {2002.09303},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2002.09303},
}
Figure