TL;DR
QUASL is a declarative framework designed to bridge the gap between natural language questions and complex Business Intelligence (BI) databases. By leveraging RDF and SPARQL to map extracted semantics to structured query models, it allows non-technical users to perform sophisticated data exploration—like range filters and personalized "top-k" queries—without knowing the underlying database schema.
What Problem Does This Paper Address?
In the world of Business Intelligence, data is stored in massive, complex "data warehouses." Currently, if a business user wants a specific insight (e.g., "Who are my top 5 middle-aged customers in Palo Alto?"), they usually have two choices: wait for the IT department to build a custom report or try to navigate a daunting technical interface.
Existing solutions like keyword search are often too simplistic; they struggle with "fuzzy" concepts like "middle-aged" (which implies a numerical range) or contextual phrases like "my city." The challenge is not just recognizing words, but understanding how those words relate to a specific database schema and ensuring the resulting query is mathematically and structurally valid.
Methods
The authors introduce QUASL, which follows a multi-stage pipeline to transform a question into a ranked list of executable queries:
- Entity Recognition: Using a combination of dictionary matching and linguistic patterns, the system identifies "artifacts" (measures, dimensions, or values). For example, "top 5" is recognized as a truncation rule, while "customers" is mapped to a dimension in the schema.
- The Parse Graph: All identified entities are stored in an RDF-based "parse graph." This graph captures the relationship between the user's words and the database metadata.
- Declarative Mapping (The Core Innovation): Instead of hard-coding rules, the authors use SPARQL graph patterns to define constraints. These patterns check if a query is valid (e.g., "Do these two dimensions actually connect to the same fact table?") and inject missing information (e.g., if a user asks for "revenue," the system might suggest "per year" to make the data meaningful).
- Query Generation: A SPARQL
CONSTRUCTclause transforms the parse graph into an intermediate query model, which is then serialized into SQL or MDX. - Scoring & Ranking: Since one question can have multiple interpretations, QUASL ranks them using three metrics:
- Confidence: How well did the text match the metadata?
- Complexity: More complex queries (using more recognized entities) are often more relevant.
- Selectivity: Queries generated by very specific, rare graph patterns are prioritized.
Datasets
The framework was evaluated using a corpus derived from ManyEyes, a collaborative visualization platform. Specifically, the authors used:
- The American Census Bureau dataset: Integrated into a test data warehouse.
- 50 "Gold Standard" Questions: These were the actual titles of the 50 most popular visualizations on ManyEyes related to the census data (e.g., "Home ownership by State").
Key Insights and Ideas
- Declarative vs. Imperative: By using SPARQL to define mapping rules, the system is highly configurable. Administrators can add custom vocabulary (like defining "rich" as a specific income range) without rewriting the core engine.
- Handling Ambiguity through Suggestion: Users often ask incomplete questions. QUASL’s ability to "propose" measures or dimensions based on schema relationships allows it to generate valid visualizations even from vague input.
- Contextualization: The framework can leverage a "User Profile" (e.g., knowing the user is in Palo Alto) to automatically inject filters into the query, making the results immediately relevant.
Key Results
The authors compared QUASL against WolframAlpha, a leading computational knowledge engine.
- Raw Performance: On the original 50 questions, QUASL significantly outperformed WolframAlpha. This is largely because QUASL was tailored to the specific census schema, whereas WolframAlpha is a general-purpose tool.
- Success@k: In terms of "Success@1" (the correct query being the top result), QUASL achieved high accuracy. When questions were slightly "updated" to use terms more familiar to the systems (e.g., changing "rich people" to "highest household income"), both systems performed comparably well, with QUASL maintaining a slight edge in the top 4 results.
Limitations and Future Work
While powerful, the authors note several areas for improvement:
- Linguistic Limitations: The current system relies heavily on exact or near-exact matches. It lacks a deep linguistic component (like WordNet) to handle synonyms (e.g., knowing "dwellings" refers to "home ownership").
- Manual Setup: Currently, the mapping between natural language patterns and the schema requires manual configuration by an administrator.
- Future Directions: The authors suggest using machine learning to improve the scoring of results based on user feedback and automating the setup process by generating patterns from existing query logs.
Summary updated