Evaluating Ordering Strategies of Star Glyph Axes

Matthias Miller, Xuan Zhang, Johannes Fuchs, Michael Blumenschein

TL;DR

This paper investigates how the arrangement of dimensions in star glyph visualizations impacts users' ability to group data. They found that ordering dimensions to create "spiky" glyphs (dissimilarity-based) significantly improves clustering performance, especially when the data contains irrelevant "clutter" dimensions.

What Problem Does This Paper Address?

Star glyphs are a popular way to visualize multi-dimensional data points, especially when comparing many of them side-by-side. However, the visual appearance of a star glyph, and thus how easily users can interpret it, is heavily influenced by the order in which its dimensions (axes) are arranged. Two main strategies exist: ordering by similarity to create smooth, uniform shapes, or by dissimilarity to create more distinct, "spiky" shapes. While spiky shapes are thought to aid in tasks like clustering, there's been a lack of empirical research directly comparing these ordering strategies, particularly in the presence of noisy or irrelevant data. This paper aims to fill that gap by evaluating which ordering strategy is more effective for data grouping tasks.

Methods

The researchers conducted a controlled user study with 15 participants. They compared two primary dimension ordering strategies:

  • Similarity-based (SIM): Aims to create homogeneous, smooth glyph shapes.
  • Dissimilarity-based (DIS): Aims to create glyphs with spikes and salient shapes.

Participants were tasked with grouping star glyphs into clusters and identifying noise points. The study evaluated four dependent variables: task completion time, quality of groups, quality of identified noise, and user confidence. A key aspect of their experimental design was the introduction of "clutter" dimensions – irrelevant attributes that don't help distinguish clusters but can obscure patterns. They tested datasets with zero, one, and two clutter dimensions.

Datasets

The study used 18 custom-generated datasets created with the PCDC tool. Each dataset contained 50 records, with 2-7 data points designated as noise. The remaining data points were divided into 2, 3, or 4 clusters. Datasets were specifically designed to include clutter dimensions, where data points were uniformly distributed, making them unhelpful for clustering. Six datasets had no clutter (0C), six had one clutter dimension (1C), and six had two clutter dimensions (2C). The generated clusters were verified using a DBSCAN algorithm.

Key Insights and Ideas

  • Clutter is the enemy of clarity: The presence of irrelevant dimensions significantly degrades users' ability to form accurate clusters, identify noise, and feel confident in their decisions. This is a general challenge in information visualization but particularly impacts star glyphs by distorting their shapes.
  • Spikes for grouping: Dissimilarity-based ordering, which emphasizes spikes and salient shapes, generally leads to better clustering performance compared to similarity-based ordering. This aligns with the intuition that distinct features help in distinguishing groups.
  • The context matters: While dissimilarity-based ordering is generally better, similarity-based ordering might be slightly faster and less prone to errors when there's no clutter. This suggests a nuanced approach might be needed depending on the data's characteristics.
  • User preference vs. performance: Interestingly, while many participants preferred the smoother shapes from similarity-based ordering, the dissimilarity-based ordering led to objectively better performance in clustering tasks, especially with clutter.

Key Results

  • Cluster Quality: Dissimilarity-based ordering (DIS) resulted in significantly higher clustering accuracy (0.69 average Jaccard index) compared to similarity-based ordering (SIM) (0.67), with a p < .05 significance. This effect was more pronounced in datasets with clutter (1C and 2C).
  • Noise Identification Quality: Clutter had a significant negative impact. For the 2C clutter condition, DIS ordering led to significantly better noise identification (0.4) than SIM (0.33).
  • Task Completion Time: While not always statistically significant, DIS ordering tended to be slightly faster, especially in cluttered datasets.
  • Confidence: Participants reported higher confidence with SIM ordering in clutter-free datasets (0C) and with DIS ordering in cluttered datasets (2C), suggesting a tendency towards DIS when the task is harder.

Limitations and Future Work

The authors acknowledge several limitations:

  • Small sample size: The number of trials (270 total) is relatively small, which can make outliers influential. They suggest repeating the study with more participants to increase statistical power.
  • Limited dimensions: The study used datasets with eight dimensions, following previous work. Future research should explore how different numbers of dimensions (especially odd numbers) affect performance.

Future work directions include:

  • Combining strategies: Investigating hybrid ordering strategies that might leverage the benefits of both SIM and DIS.
  • Rotation's influence: Exploring how rotating entire glyphs affects grouping quality.
  • Data-task-ordering alignment: Developing techniques to automatically select the most effective ordering strategy based on the data characteristics and the specific analysis task.
  • User-driven vs. automatic: Comparing automatic ordering strategies with those manually set by domain experts.

Summary updated

Abstract

Star glyphs are a well-researched visualization technique to represent multi-dimensional data. They are often used in small multiple settings for a visual comparison of many data points. However, their overall visual appearance is strongly influenced by the ordering of dimensions. To this end, two orthogonal categories of layout strategies are proposed in the literature: order dimensions by similarity to get homogeneously shaped glyphs vs. order by dissimilarity to emphasize spikes and salient shapes. While there is evidence that salient shapes support clustering tasks, evaluation, and direct comparison of data-driven ordering strategies has not received much research attention. We contribute an empirical user study to evaluate the efficiency, effectiveness, and user confidence in visual clustering tasks using star glyphs. In comparison to similarity-based ordering, our results indicate that dissimilarity-based star glyph layouts support users better in clustering tasks, especially when clutter is present.
BibTeX
@inproceedings{miller2019evaluating,
  author = {Matthias Miller and Xuan Zhang and Johannes Fuchs and Michael Blumenschein},
  title = {{Evaluating Ordering Strategies of Star Glyph Axes}},
  booktitle = {IEEE VIS (Short Papers)},
  year = {2019},
  url = {https://openreview.net/forum?id=2vN0wgVRSW},
}
Figure