DIG

Yiru Chen, Jeffrey Tao, Eugene WuarXiv 2023 · DOI · arXiv

Abstract

Building interactive data interfaces is hard because the design of an interface depends on the data processing needs for the underlying analysis task, yet we do not have a good representation for analysis tasks. To fill this gap, this paper advocates for a Data Interface Grammar (DIG) as an intermediate representation of analysis tasks. We show that DIG is compatible with existing data engineering practices, compact to represent any analysis, simple to translate into an interface design, and amenable to offline analysis. We further illustrate the potential benefits of this abstraction, such as automatic interface generation, automatic interface backend optimization, tutorial generation, and workload generation.

Introduction

The design of interactive data interfaces is currently a fragmented and manual process. Data engineers focus on writing efficient queries, while UI designers focus on widgets like sliders and dropdowns. These two domains are often bridged by "string-gluing"—concatenating SQL fragments in application code—which is error-prone, difficult to optimize, and hard to maintain. Existing abstractions like predefined or parameterized queries are either too rigid or only handle simple filter changes. This paper introduces the Data Interface Grammar (DIG) as a "middle ground" that compactly represents analysis tasks while remaining machine-readable for both databases and user interfaces.

The Data Interface Grammar (DIG)

DIG extends Parsing Expression Grammars (PEG) with data-specific annotations to represent parts of a query that a user can change.

  • Choice Variables: Non-terminals represent user-adjustable parts of a query. For instance, a variable t might represent a choice between data sources, while val represents a slider's value.
  • Typed Domains: DIG supports "Predicate Domains" (e.g., integers) and "Query Domains" (e.g., lists fetched from the database).
  • Constraints: Developers can define logic (e.g., start_date <= end_date) that the UI must automatically enforce.
Figure 1: Mapping a drought insurance interface to a DIG grammar. The grammar defines choice variables for data sources and range values.

Expressiveness and Recursion

Unlike many UI frameworks, DIG can handle recursive rules, allowing it to represent complex interfaces like nested query builders. This enables structural changes to the query, such as switching between a table name and a subquery in a FROM clause, which standard parameterized queries cannot handle.

Figure 2: A recursive DIG grammar representing a complex query builder. This illustrates how structural changes in a UI map to grammar rules.

Key Insights and Applications

A core insight of DIG is that interacting with a UI is mathematically equivalent to navigating a grammar. Once all choices are made, the grammar reduces to a valid, executable query. This formalization enables several benefits:

  • Physical Visualization Design (PVD): Since DIG defines the universe of possible queries, the system can perform offline analysis to optimize performance, such as deciding where to build indices to meet latency goals.
  • Automatic Tutorials: By tracking the UI state as grammar bindings, the system can calculate the sequence of interactions needed to reach a specific state.
  • Progressive Design: Developers can start with simple text inputs and gradually refine them into specialized widgets without rewriting underlying logic.

Conclusion

DIG provides a robust intermediate representation that decouples data processing logic from UI design. By formalizing analysis tasks, it enables automated interface generation and backend optimization, bridging the gap between data engineering and interactive exploration.

Summary updated

Abstract

Building interactive data interfaces is hard because the design of an interface depends on the data processing needs for the underlying analysis task, yet we do not have a good representation for analysis tasks. To fill this gap, this paper advocates for a Data Interface Grammar (DIG) as an intermediate representation of analysis tasks. We show that DIG is compatible with existing data engineering practices, compact to represent any analysis, simple to translate into an interface design, and amenable to offline analysis. We further illustrate the potential benefits of this abstraction, such as automatic interface generation, automatic interface backend optimization, tutorial generation, and workload generation.
BibTeX
@misc{chen2023dig,
  author = {Yiru Chen and Jeffrey Tao and Eugene Wu},
  title = {{DIG}},
  year = {2023},
  eprint = {2307.03336},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2307.03336},
}
Figure