Natural Language Question Answering and Analytics for Diverse and Interlinked Datasets

Dezhao Song, Frank Schilder, Charese Smiley, Chris BrewNAACL 2015

TL;DR

This paper introduces a Natural Language Interface (NLI) that allows users to query complex, interlinked datasets using everyday language. It goes beyond simple answer retrieval by offering auto-suggestions to help users formulate questions and providing dynamic, rich analytics on the results, making data more accessible and understandable for non-technical users.

What Problem Does This Paper Address?

Knowledge workers, like physicians or financial analysts, often struggle to access information locked within complex knowledge bases (KBs). This is because they need to learn specialized query languages (e.g., SQL, SPARQL), which are constantly evolving. This steep learning curve prevents many users from effectively utilizing the vast amounts of data available. Existing Natural Language Interfaces (NLIs) often require users to craft perfect, well-formed questions and typically return raw lists of answers, which can be overwhelming and lack context. The authors aim to bridge this gap by creating a user-friendly NLI that simplifies question formulation and enhances data understanding through intelligent suggestions and dynamic analytics.

Methods

The core of the system is a feature-based context-free grammar (FCFG) that is derived from interlinked datasets. This grammar allows the system to parse natural language questions into an intermediate First Order Logic (FOL) representation. This FOL representation is then translated into executable queries for various underlying KBs, including SQL and SPARQL, using specialized translators.

A key innovation is the auto-suggestion feature. Unlike traditional query completion based on logs, this system leverages the linguistic constraints encoded in its grammar. It employs a left-corner parsing approach to suggest possible continuations for a partial query, ensuring grammatical correctness and semantic relevance.

For query understanding, the grammar uses domain-specific features on non-terminal syntactic nodes to constrain parses and ensure unambiguous interpretations. This includes handling prepositional phrase attachment preferences to resolve ambiguities.

Finally, the system goes beyond just retrieving answers. It generates dynamic analytics on the result sets, including descriptive analytics, comparisons across dimensions, and trend analysis using exponential smoothing. It also integrates with a large news corpus to perform sentiment analysis and frequency counts for entities, providing richer contextual insights.

Datasets

The paper mentions that the underlying knowledge base contains about 1 million entities and 12 million relationships. The system is currently focused on the domains of Drugs, Organizations, Patents, People, Finance, and News. They also leverage a large news corpus (14 million articles and 147 million sentences) for additional analytics. No new benchmarks are introduced; rather, the system is designed to work with existing interlinked datasets.

Key Insights and Ideas

  • Bridging the gap for non-technical users: The most significant insight is the realization that simplifying question formulation (via auto-suggestion) and enhancing answer presentation (via dynamic analytics) is crucial for making complex data accessible to a wider audience.
  • Grammar as the backbone: Using a feature-based grammar derived from interlinked datasets provides a robust and adaptable foundation for understanding natural language queries across different domains and KB types.
  • Beyond raw answers: The paper highlights the value of providing context and deeper insights through analytics. Simply returning a list of entities is often insufficient; users need tools to understand the relationships, trends, and sentiment surrounding the data.
  • Flexibility in query translation: Translating to an intermediate FOL representation before generating SQL or SPARQL offers flexibility to adapt to new query languages and choose the most appropriate one for a given use case.

Key Results

The paper doesn't present specific quantitative results like accuracy scores or performance benchmarks. However, it demonstrates the system's capabilities through examples:

  • Auto-suggestion: The system can suggest relevant continuations for partial queries, guiding users towards complete and meaningful questions.
  • Query translation: It successfully translates natural language questions into executable SQL and SPARQL queries.
  • Dynamic analytics: The system generates visualizations like pie charts (e.g., company market share for hypertension drugs) and trend graphs (e.g., news mentions and sentiment over time), providing richer insights than simple answer lists.
  • Handling ambiguity: The grammar effectively handles complex queries with multiple modifiers and resolves ambiguities in prepositional phrase attachments.

Limitations and Future Work

Limitations: * The paper does not provide quantitative performance metrics or accuracy evaluations for the NLI. * The system's current focus is on a limited set of domains.

Future Work: * Expand the system to cover more domains. * Develop and provide more complex types of analytics. * Conduct a thorough evaluation of the system's performance and user satisfaction.

Summary updated

BibTeX
@inproceedings{song2015natural,
  author = {Dezhao Song and Frank Schilder and Charese Smiley and Chris Brew},
  title = {{Natural Language Question Answering and Analytics for Diverse and Interlinked Datasets}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2015},
}
Figure