Graphical Perception of Value Distributions: An Evaluation of Non-Expert Viewers' Data Literacy

Arkaitz Zubiaga, Brian Mac NameeJ. Community Informatics 2016 · DOI · OpenReview

Abstract

While data visualization is a cornerstone of modern communication, design choices are frequently driven by aesthetic preference or expert intuition rather than empirical evidence of how general audiences decode information. This research evaluates the "graphical perception" of non-experts—their ability to accurately interpret value distributions across various chart types. The findings highlight a significant "data literacy gap" and establish histograms as the superior tool for communicating frequency data to the public.

Introduction

In an era of data ubiquity, the challenge is no longer just presenting data, but ensuring it is understood. This paper addresses two primary gaps in visualization research: the literacy gap (how well non-experts understand different distribution formats) and the comparison challenge (how best to display two groups simultaneously for accurate interpretation).

Methods

The researchers conducted two large-scale user studies via Amazon Mechanical Turk to reach a diverse, non-expert audience.

Study 1 presented participants with five distinct chart types: bar charts of averages, bee swarms, boxplots, stacked bar charts, and histograms. Participants were asked to validate statements about the data, such as identifying where the majority of data points fell. A critical inclusion in this study was the "impossible to tell" option, designed to measure whether users recognized when a visualization lacked the necessary information to answer a specific question.

Comparison of chart types evaluated in Study 1, including bar charts, bee swarms, boxplots, histograms, and stack charts.

Study 2 narrowed the focus to a head-to-head comparison between histograms and density traces. It tested both single and dual distributions using various layouts, including overlapped, mirrored, interspersed (side-by-side bars), stacked, and cumulative arrangements. Performance was measured by accuracy (responses within 10% of ground truth), error rates, and response times.

Datasets

To ensure robust results, the studies utilized both synthetic and real-world data:

  • Study 1 Datasets: Four artificially generated datasets representing common scenarios: customer ages (unimodal), sports center member ages (uniform), resident salaries (skewed), and student exam scores (bimodal).
  • Study 2 Datasets: Real-world economic data from the Vietnamese Living Standard Study (VLSS), specifically household per capita expenditure for rural versus urban populations. This provided the complex, skewed distributions typical of real-world scenarios.
The four synthetic datasets used in Study 1 to represent various distribution shapes: customer ages, member ages, resident salaries, and exam scores.

Key Insights

The Data Literacy Gap

A striking finding was the "Don't Know What They Don't Know" problem. When presented with a chart that lacked the information required to answer a question (e.g., using a bar chart of averages to answer a question about distribution), participants rarely selected "impossible to tell." Instead, they made confident but incorrect inferences. This suggests that non-experts are highly susceptible to misinformation when provided with oversimplified visualizations.

The Superiority of Histograms

Histograms emerged as the "Goldilocks" of distribution charts. While density traces are mathematically smooth and bee swarms show individual points, histograms provide the optimal balance for human cognition. The discrete "binning" of data makes it significantly easier for viewers to quantify frequencies compared to the continuous curves of a density trace.

Layout Matters for Comparison

When comparing two groups, "interspersed" histograms—where bars for different groups sit side-by-side within each bin—were the clear winners. Despite the thinner bars, the lack of occlusion (overlap) makes the comparative data much easier to process than mirrored or stacked versions.

Key Results

  • Top Performance: In Study 1, histograms achieved a dominant 96.9% accuracy rate, followed by bee swarms at 90.6%. Conversely, boxplots (56.3%) and stacked bars (43.8%) performed poorly for general distribution questions.
  • Quantification Accuracy: In Study 2, histograms (74.8% average accuracy) outperformed density traces (70.3%). While density traces were effective at showing general trends, they failed when users needed to estimate specific numerical values.
  • Comparison Layouts: Interspersed histograms reached 85% accuracy for dual distributions, significantly higher than overlapped (77%), mirrored (72%), or stacked (78%) versions.
  • The Failure of Cumulative Charts: Cumulative histograms proved nearly impossible for non-experts to decode, resulting in a dismal 21% accuracy rate.

Conclusion and Future Work

Histograms remain the gold standard for communicating data distributions to non-experts. However, the study also highlights the importance of "scaffolding"; accuracy dropped significantly when values did not align with X-axis tick marks. Future research should explore these perceptions across different cultural and demographic groups, and investigate how the number of compared groups (e.g., comparing five populations) affects the relative efficacy of histograms versus density traces.

Summary updated

Abstract

An ability to understand the outputs of data analysis is a key characteristic of data literacy and the inclusion of data visualisations is ubiquitous in the output of modern data analysis. Several aspects still remain unresolved, however, on the question of choosing data visualisations that lead viewers to an optimal interpretation of data. This is especially true when audiences have differing degrees of data literacy. In this paper we describe two user studies on perception from data visualisations, in which we measured the ability of participants to validate statements about the distributions of data samples visualised using different chart types. In the first user study, we find that histograms are the most suitable chart type for illustrating the distribution of values for a variable. We contrast our findings with previous research in the field, and posit three main issues identified from the study. Most notably, however, we show that viewers struggle to identify scenarios in which a chart simply does not contain enough information to validate a statement about the data that it represents. In the follow-up study, we ask viewers questions about quantification of frequencies, and identification of most frequent values from different types of histograms and density traces showing one or two distributions of values. This study reveals that viewers do better with histograms when they need to quantify the values displayed in a chart. Among the different types of histograms, interspersing the bars of two distributions in a histogram leads to the most accurate perception. Even though interspersing bars makes them thinner, the advantage of having both distributions clearly visible pays off. The findings of these user studies provide insight to assist designers in creating optimal charts that enable comparison of distributions, and emphasise the importance of using an understanding of the limits of viewers’ data literacy to design charts effectively.
BibTeX
@article{zubiaga2016graphical,
  author = {Arkaitz Zubiaga and Brian Mac Namee},
  title = {{Graphical Perception of Value Distributions: An Evaluation of Non-Expert Viewers' Data Literacy}},
  journal = {J. Community Informatics},
  year = {2016},
  doi = {10.15353/joci.v12i3.3282},
}
Figure