Abstract
In this paper, we test whether Multimodal Large Language Models (MLLMs) can match human-subject performance in tasks involving the perception of properties in network layouts. Specifically, we replicate a human-subject experiment about perceiving quality (namely stress) in network layouts using GPT-4o and Gemini-2.5. Our experiments show that giving MLLMs exactly the same study information as trained human participants results in a similar performance to human experts and exceeds the performance of untrained non-experts. Additionally, we show that prompt engineering that deviates from the human-subject experiment can lead to better-than-human performance in some settings. Interestingly, like human subjects, the MLLMs seem to rely on visual proxies rather than computing the actual value of stress, indicating some sense or facsimile of perception. Explanations from the models provide descriptions similar to those used by the human participants (e.g., even distribution of nodes and uniform edge lengths).
Introduction
In network science, "stress" is a critical metric for layout quality, measuring how well the physical distances between nodes in a drawing reflect their actual graph-theoretic distances. While stress is mathematically well-defined, calculating it requires access to the underlying graph data. Humans, however, can often perceive or "feel" stress just by looking at a node-link diagram. Evaluating these visualizations typically requires expensive human-subject studies. This research investigates whether state-of-the-art Multimodal Large Language Models (MLLMs) possess "visualization literacy" and can perceive these aesthetic properties similarly to humans, potentially automating the evaluation of new layout algorithms.
Methodology
The researchers conducted a rigorous replication of a 2024 study by Mooney et al., presenting GPT-4o and Gemini-2.5 with 216 pairs of Erdős–Rényi random networks (10, 25, and 50 nodes). The models were tasked with identifying which layout in a pair had lower Kruskal stress.
Four prompting conditions were evaluated:
- Expert: Minimal instructions, mimicking a human expert who already knows the criteria.
- Untrained: Provided basic definitions of nodes, edges, and stress, but no practice.
- Trained: Provided definitions plus nine training examples with correct/incorrect feedback.
- Tuned: An optimized prompt using "Chain of Thought" reasoning, directing the model toward specific visual proxies like edge crossings and node uniformity.
To ensure statistical rigor, a bootstrapping approach (10,000 resamples) was used to compare MLLM performance directly against the original human-subject data, which included Untrained Novices, Trained Novices, and Experts.
Results
The study revealed that MLLMs do not calculate mathematical stress but instead rely on a "facsimile of perception." Reasoning logs indicated that models focus on edge crossings, node distribution, and edge length uniformity—the same features used by human experts.
Key quantitative findings include:
- Overall Accuracy: In the "Trained" setting, GPT-4o (77.3%) and Gemini-2.5 (81.5%) performed comparably to human experts (78%) and significantly better than untrained humans (68%).
- Tuned Performance: With optimized prompts, MLLMs achieved near-perfect accuracy on diagrams with large stress differences, outperforming human experts by roughly 10% on positive-difference pairs.
- Heuristic Correlation: Logistic regression confirmed that edge length deviation was the strongest predictor of a model's choice, followed by node uniformity and edge crossings.
However, MLLMs struggled when presented with diagrams of nearly identical stress. Unlike humans, who often chose a "they are the same" option when uncertain, MLLMs were overconfident, leading to near-zero accuracy in that specific subset.
Discussion and Conclusion
The results suggest that MLLMs can effectively serve as "pilot participants" for visualization research, provided they are given appropriate context. The "Expert" paradox—where models performed worse with minimal context than human experts—highlights the importance of explicit persona setting to activate latent visualization knowledge.
Future work should address model volatility and explore whether MLLMs can handle lower-level tasks like path-following or identifying clusters in larger datasets. Additionally, investigating if MLLMs can be deceived by "adversarial" layouts that appear aesthetically pleasing but misrepresent data remains a critical area for exploration.
Summary updated