Knowledge graphs, semantic networks, and ontologies provide a powerful way to model complex, heterogeneous data. By representing information as interconnected nodes and edges, frameworks like the Resource Description Framework (RDF) allow organizations to link everything from clinical trials to industrial sensor data.
However, extracting value from these structures traditionally requires writing formal queries in languages like SPARQL or SQL. For domain experts—such as geologists, clinicians, or social scientists—this creates a severe technical bottleneck. They know what data they need, but they lack the programming expertise to express complex graph traversals, joins, and logical constraints.
To bridge this gap, a rich body of research focuses on Visual Query Formulation (VQF). The core intuition behind VQF is to shift the user's cognitive load from recalling syntax to recognizing concepts. Instead of forcing users to write textual code, VQF frameworks translate intuitive visual interactions—such as clicking facets, navigating trees, or sketching shapes—directly into mathematically rigorous graph queries.
Shifting Cognitive Load via Faceted Navigation
One of the most successful paradigms in VQF adapts the familiar "e-commerce filtering" experience to complex graph structures. In traditional databases, a faceted search filters flat rows. In a knowledge graph, however, facets represent properties and relationships that span multiple interconnected entities.
When a user interacts with a faceted interface, the underlying system treats each selection as a step in a graph traversal. For example, Slepicka et al. demonstrate how a faceted search interface can transform a billion-node knowledge graph into a manageable filtering process for non-technical investigators. Under the hood, each user action is dynamically compiled into specific fragments of a formal query language.
As explored by Cuenca Grau et al., these interfaces can support complex logical operations (such as AND-UNION sets) by mapping visual selections directly to join-heavy SPARQL queries. By transforming formal ontologies into interactive "navigation graphs," systems like the one proposed by Soylu et al. guide users through query construction step-by-step. This ensures that every visual interaction is guaranteed to produce a syntactically valid and executable query, shielding the user from the underlying schema complexity.
Structural Simplification: Trees and Summaries
While graphs are highly expressive, visualizing a dense network can overwhelm users. To mitigate this, several approaches project complex graph topologies into simpler, hierarchical structures like interactive trees or schema summaries.
In the VXMLR system, Lü et al. present hierarchical schemas as interactive trees, allowing users to construct complex path expressions through direct manipulation rather than manual coding. Similarly, Yu et al. introduce the concept of a "schema summary," which abstracts away the full complexity of an intricate database schema into a high-level representation that guides structure-free keyword queries.
This structural simplification is also powerful for debugging automated systems. For instance, in Knowledge Base Question Answering (KBQA), natural language parsers often generate incorrect logical forms. Yao et al. address this by representing functional query programs as interactive graphical trees. Users can visually manipulate flowcharts and fill slots to correct errors in the underlying logic, bridging the gap between natural language intent and formal execution.
Query-by-Example and Multimodal Inputs
Another intuitive approach bypasses schema exploration entirely by allowing users to define queries using familiar document structures or visual gestures.
Muhao Chen et al. introduce a query-by-example interface built directly into Wikipedia-style infoboxes. Users simply fill in fields within a familiar visual template, and the system automatically maps these inputs to variables and constraints in a temporal SPARQL query, completely hiding the underlying database identifiers.
For data types where text and tables fail, multimodal inputs like sketching provide a direct bridge for human intuition. In the domain of time-series and visual data exploration, Siddiqui et al. develop Zenvisage, which allows users to literally "sketch" a desired trend or pattern on a canvas. The system then translates this fuzzy visual shape into a precise query language to find matching patterns across massive datasets. Similarly, in architectural databases, Langenhan et al. translate freehand spatial sketches into structured graph queries that represent spatial adjacencies.
For application development, Kagal et al. demonstrate that block-based programming interfaces can abstract away RDF generation and SPARQL syntax entirely, allowing non-programmers to assemble logic visually using drag-and-drop blocks. In multi-modal datasets, Wood shows that drag-and-drop interfaces can automatically handle underlying table joins correctly, preventing user errors when combining disparate data types like phenotypic and imaging data.
The Compilation Layer: From Pixels to Logic
The true power of visual query formulation lies in its backend compilation layer. A visual interface is only useful if it can faithfully preserve the expressive power of the underlying formal languages without exposing their complexity.
In the industrial platform Optique, described by Kharlamov et al. and Giese et al., a graph-based visual interface translates user interactions into tree-shaped conjunctive queries. These are then automatically compiled into highly optimized SQL or SPARQL queries that run against massive, distributed databases.
This translation mechanism applies equally to analytical operations. For example, Hose details a translation layer that maps high-level visual business intelligence actions—such as slicing, dicing, and rolling up data—directly into equivalent SPARQL queries. By encapsulating these complex graph queries into standardized interfaces or parameterized templates, as also seen in work by Dibowski et al. and Emonet et al., the technical barriers of semantic technologies are completely hidden from the end-user workflow.
Key Foundational Work
VisualMOQL: The DISIMA Visual Query Language (1999) — contributed 2 concept elements to this direction.