Exploring MLLMs Perception of Network Visualization Principles

Jacob Miller, Markus Wallinger, Ludwig Felder, Timo Brand, Henry Förster, Johannes Zink, Chunyang Chen, Stephen Kobourov

Abstract

In this paper, we test whether Multimodal Large Language Models (MLLMs) can match human-subject performance in tasks involving the perception of properties in network layouts. Specifically, we replicate a human-subject experiment about perceiving quality (namely stress) in network layouts using GPT-4o and Gemini-2.5. Our experiments show that giving MLLMs exactly the same study information as trained human participants results in a similar performance to human experts and exceeds the performance of untrained non-experts. Additionally, we show that prompt engineering that deviates from the human-subject experiment can lead to better-than-human performance in some settings. Interestingly, like human subjects, the MLLMs seem to rely on visual proxies rather than computing the actual value of stress, indicating some sense or facsimile of perception. Explanations from the models provide descriptions similar to those used by the human participants (e.g., even distribution of nodes and uniform edge lengths).

Introduction

In network science, "stress" is a critical metric for layout quality, measuring how well the physical distances between nodes in a drawing reflect their actual graph-theoretic distances. While stress is mathematically well-defined, calculating it requires access to the underlying graph data. Humans, however, can often perceive or "feel" stress just by looking at a node-link diagram. Evaluating these visualizations typically requires expensive human-subject studies. This research investigates whether state-of-the-art Multimodal Large Language Models (MLLMs) possess "visualization literacy" and can perceive these aesthetic properties similarly to humans, potentially automating the evaluation of new layout algorithms.

Methodology

The researchers conducted a rigorous replication of a 2024 study by Mooney et al., presenting GPT-4o and Gemini-2.5 with 216 pairs of Erdős–Rényi random networks (10, 25, and 50 nodes). The models were tasked with identifying which layout in a pair had lower Kruskal stress.

Four prompting conditions were evaluated:

  1. Expert: Minimal instructions, mimicking a human expert who already knows the criteria.
  2. Untrained: Provided basic definitions of nodes, edges, and stress, but no practice.
  3. Trained: Provided definitions plus nine training examples with correct/incorrect feedback.
  4. Tuned: An optimized prompt using "Chain of Thought" reasoning, directing the model toward specific visual proxies like edge crossings and node uniformity.
MLLM responses showing visual proxies for stress perception.

To ensure statistical rigor, a bootstrapping approach (10,000 resamples) was used to compare MLLM performance directly against the original human-subject data, which included Untrained Novices, Trained Novices, and Experts.

Results

The study revealed that MLLMs do not calculate mathematical stress but instead rely on a "facsimile of perception." Reasoning logs indicated that models focus on edge crossings, node distribution, and edge length uniformity—the same features used by human experts.

Accuracy comparison across different prompting conditions and human baselines.

Key quantitative findings include:

  • Overall Accuracy: In the "Trained" setting, GPT-4o (77.3%) and Gemini-2.5 (81.5%) performed comparably to human experts (78%) and significantly better than untrained humans (68%).
  • Tuned Performance: With optimized prompts, MLLMs achieved near-perfect accuracy on diagrams with large stress differences, outperforming human experts by roughly 10% on positive-difference pairs.
  • Heuristic Correlation: Logistic regression confirmed that edge length deviation was the strongest predictor of a model's choice, followed by node uniformity and edge crossings.

However, MLLMs struggled when presented with diagrams of nearly identical stress. Unlike humans, who often chose a "they are the same" option when uncertain, MLLMs were overconfident, leading to near-zero accuracy in that specific subset.

Discussion and Conclusion

The results suggest that MLLMs can effectively serve as "pilot participants" for visualization research, provided they are given appropriate context. The "Expert" paradox—where models performed worse with minimal context than human experts—highlights the importance of explicit persona setting to activate latent visualization knowledge.

Future work should address model volatility and explore whether MLLMs can handle lower-level tasks like path-following or identifying clusters in larger datasets. Additionally, investigating if MLLMs can be deceived by "adversarial" layouts that appear aesthetically pleasing but misrepresent data remains a critical area for exploration.

Summary updated

Abstract

In this paper, we test whether Multimodal Large Language Models (MLLMs) can match human-subject performance in tasks involving the perception of properties in network layouts. Specifically, we replicate a human-subject experiment about perceiving quality (namely stress) in network layouts using GPT-4o and Gemini-2.5. Our experiments show that giving MLLMs exactly the same study information as trained human participants results in a similar performance to human experts and exceeds the performance of untrained non-experts. Additionally, we show that prompt engineering that deviates from the human-subject experiment can lead to better-than-human performance in some settings. Interestingly, like human subjects, the MLLMs seem to rely on visual proxies rather than computing the actual value of stress, indicating some sense or facsimile of perception. Explanations from the models provide descriptions similar to those used by the human participants (e.g., even distribution of nodes and uniform edge lengths).
BibTeX
@article{miller2026exploring,
  author = {Jacob Miller and Markus Wallinger and Ludwig Felder and Timo Brand and Henry Förster and Johannes Zink and Chunyang Chen and Stephen Kobourov},
  title = {{Exploring MLLMs Perception of Network Visualization Principles}},
  journal = {IEEE Trans. Vis. Comput. Graph.},
  year = {2026},
  doi = {10.1109/tvcg.2026.3681983},
}
Figure