Natural Language Aided Visual Query Building for Complex Data Access

AAAI 2010

TL;DR

This paper introduces TAICHI, a novel system that augments traditional visual query interfaces with natural language (NL) capabilities. TAICHI allows users to seamlessly blend NL expressions with visual query building, significantly improving usability and reducing the knowledge gap for complex data access.

What Problem Does This Paper Address?

Accessing data stored in complex databases often requires users to be proficient in query languages like SQL or navigate intricate visual query interfaces. While visual query interfaces offer a more intuitive approach than raw SQL, they can be rigid and demand a deep understanding of the underlying database schema and data values. This "knowledge gap" can be a significant barrier for business users who need to make data-driven decisions but lack database expertise. Existing natural language interfaces, while promising, often struggle with the inherent ambiguity and diversity of human language, leading to interpretation challenges. This paper tackles the problem of bridging this gap by creating a system that leverages the strengths of both visual and natural language interfaces.

Methods

TAICHI's core innovation lies in its hybrid approach to query building. It combines a visual query composer with an NL interpreter.

  • Visual Query Composer: This component is responsible for generating and updating the visual representation of a query graph, which serves as an intermediate query representation. It aims to maintain "visual momentum" by showing incremental changes to the query, making the process feel more fluid. It uses a rule-based graphics generation approach to map query graph elements to visual metaphors.
  • NL Interpreter: This component translates natural language expressions into query graph elements. A key aspect of TAICHI's NL interpreter is its reliance on the restricted visual query context. Instead of trying to interpret NL in isolation, TAICHI uses the existing visual query elements and the user's interaction history to disambiguate and refine NL interpretations. This context-aware approach significantly improves accuracy.
  • TAICHI Architecture: The system comprises three main components: the NL interpreter, the visual query composer, and a dispatcher that communicates with a web client. It also leverages several knowledge sources, including domain data from a database server, application metadata (like a semantic dictionary), and interaction history.

TAICHI's NL processing involves a six-step process: morphological analysis, term identification, semantic tagging, semantic unification, relation analysis, and context analysis. The paper highlights how the restricted visual query context helps in disambiguation, for example, by filtering semantic tags based on the current concept being edited.

Datasets

Not discussed in the paper. The paper uses examples from real-world applications, specifically a real-estate application and a trade management application, to illustrate its functionality.

Key Insights and Ideas

  • Context is King for NL Interpretation: The most significant insight is that leveraging the restricted visual query context dramatically improves the accuracy of natural language interpretation. By understanding what the user is currently doing visually, TAICHI can make much more informed decisions about the meaning of NL input.
  • Bridging the Knowledge Gap: TAICHI effectively addresses the "knowledge gap" by allowing users to express complex data needs using NL when they don't know how to map them to the visual interface or database schema. This makes complex data access more accessible.
  • Seamless Integration: The system is designed to feel like a natural extension of a visual query interface, rather than a separate NL tool. Users can fluidly switch between visual and NL input, with the system automatically updating the visual representation.
  • Incremental Query Building: The focus on maintaining "visual momentum" through incremental updates makes the query building process less daunting and more intuitive, especially for complex queries.

Key Results

The paper presents a user evaluation comparing the full TAICHI system with a "TAICHI-visual" (visual query only) version.

  • Effectiveness: TAICHI significantly reduced elapsedTime (time taken to complete tasks) compared to TAICHI-visual (209 seconds vs. 352 seconds), with a statistically significant difference (p<0.048).
  • Expressiveness: TAICHI was rated as more expressive (4.67 vs. 4.00 on a 1-5 scale), with users finding it easier to express complex requests in NL when they didn't know how to do so visually. This difference was statistically significant (p<0.035).
  • Reliability: The NL interface was found to be reliable, with 92.9% of NL queries interpreted correctly. Errors primarily occurred when NL was used as a standalone query without visual context.
  • User Preference: Participants successfully completed all tasks with both systems, but TAICHI offered a more efficient and expressive experience.

Limitations and Future Work

  • NL Standalone Queries: The paper acknowledges that NL interpretation accuracy decreases when NL is used as a standalone query without the benefit of visual context. Future work could focus on improving standalone NL interpretation.
  • Ambiguity Handling: While context helps, the paper notes that ambiguity can still arise, leading to disambiguation panels. Further research could explore more sophisticated methods for resolving ambiguities.
  • Domain Specificity: The NL interpreter relies on semantic dictionaries and domain knowledge. Expanding these resources and improving the generalization capabilities of the NL interpreter would be beneficial.
  • Error Correction: While NL error correction is described as easy due to the conversion to a visual query, further exploration of more direct NL-based error correction mechanisms could be valuable.

Summary updated

BibTeX
@inproceedings{2010natural,
  title = {{Natural Language Aided Visual Query Building for Complex Data Access}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2010},
}
Figure