Abstract
Analysts require interactive speeds for exploratory data analysis (EDA), yet modern big data systems often impose significant latency. While Approximate Query Processing (AQP) via sampling offers a path to responsiveness, it frequently compromises user trust and accuracy. We introduce optimistic visualization, a user experience framework that enables analysts to explore approximate results interactively while providing robust mechanisms for background verification and error recovery. Our prototype system, Pangloss, implements these principles. Through a laboratory study with expert visualizers and case studies at Microsoft, we demonstrate that this "trust, but verify" approach enhances analyst confidence and maintains creative flow without sacrificing precision.
The Challenge of Big Data Exploration
Exploratory Data Analysis is inherently iterative. To maintain a "flow state," analysts typically need query results in under a second. However, as datasets scale to hundreds of millions or billions of rows, traditional database engines can take minutes to return a single result, stalling the exploration process.
Approximate Query Processing (AQP) addresses this speed bottleneck by using data sampling to provide rapid "best guess" answers. However, AQP introduces a trust deficit: analysts fear that samples might miss critical outliers or introduce statistical bias. Furthermore, "progressive" visualization systems that update continuously can be distracting, as "dancing bars" fluctuate while more data is scanned.
Optimistic Visualization and Pangloss
To bridge the gap between speed and trust, we propose Optimistic Visualization. This pattern allows the system to:
- Show an approximation instantly using a small data sample.
- Allow the user to "remember" a view, triggering a full, precise query in the background.
- Notify the user when the precise result is ready, enabling a direct comparison between the initial estimate and the ground truth.
Pangloss, our prototype implementation, sits atop the Sample+Seek AQP engine. A core innovation in Pangloss is the concept of Distribution Uncertainty. Rather than relying solely on per-group confidence intervals (error bars), Pangloss calculates a global metric representing how much the overall shape of the visualization might deviate from the true distribution.
Evaluation and Real-World Application
We evaluated Pangloss using several massive real-world datasets, including 170 million rows of BTS flight data, software telemetry, search engine logs (nearly 1 billion rows), and the Twitter firehose.
Key Performance and User Results
- Speed vs. Precision: Pangloss delivered approximate results for 170 million rows in approximately 100ms (with 1–2% uncertainty), compared to roughly one minute for a full SQL query on identical hardware.
- Workflow Integration: Analysts treated approximations as "drafts" to debug queries. If a fast view looked incorrect, they could adjust filters immediately rather than waiting for a slow, precise query to confirm a typo.
- The "Ship Room" Requirement: While comfortable with samples for exploration, analysts required precise data for high-stakes decision-making. Optimistic visualization supports this by providing interactive speed for the analyst and verifiable precision for the boardroom.
Insights and Future Directions
A significant finding was the value of identifying "missing" data. Sampling often misses the "long tail"; Pangloss highlights newly discovered groups in the precise result, helping analysts understand the limitations of their initial samples.
Despite its success, Pangloss has limitations. Currently, users must manually trigger precise queries, and the system struggles with non-linear transformations like logarithmic scales. Future work will explore automated verification triggers and support for more complex visualization types beyond bar charts and heatmaps. By treating approximations as a starting point rather than the final answer, optimistic visualization provides a scalable path for interactive big data exploration.
Summary updated