Do Transaction Prices Reflect Visual Preference? A Diagnostic Study with Multimodal Language Models

Fengxian Ji, Jingpu Yang, Fan Zhang, Junhong Liang, Qian Jiang, Yuke Li, Jinri Guo, Rongxin Zhu, Zhuohan XiearXiv 2026 · arXiv

TL;DR

This paper introduces MarketImage, a diagnostic framework designed to determine if online transaction prices can serve as a low-cost proxy for human visual preference. The authors find that transaction prices are a noisy and often contradictory signal: only about a third of non-tied price directions actually align with human visual rankings, and complex market-inspired elicitation methods (like simulated auctions) often degrade the performance of Multimodal Language Models (MLMs) rather than improving it.

What Problem Does This Paper Address?

Training and evaluating generative vision models currently depends on expensive human annotations or reward models. In contrast, market transaction prices are abundant and reflect "real-world" value. However, a higher price doesn't always mean an image is "better" to the human eye; it might just be rarer, come from a famous artist, or have been sold during a period of high demand.

The authors tackle the identifiability challenge: distinguishing between visual preference signals and non-visual market noise (like provenance or competition). They investigate whether transaction prices can be used as "ground truth" labels for visual preference and whether modern MLMs can recover these signals through various elicitation interfaces.

Methods

The researchers developed the MarketImage framework, which consists of three core stages:

  1. MarketImageArt: A new dataset pairing real auction prices with price-blind human rankings.
  2. MarketImageLens: A diagnostic protocol that breaks down image pairs into three categories: price-human agreement, price-human conflict, and price ties. It also audits for "presentation-order effects"—the tendency for humans or models to favor items based on their position in a list.
  3. MarketValuator: A suite of six elicitation methods used to test MLMs (like GPT-5.5 and Qwen3-VL). Instead of just asking a model "which image is better?", they tested more complex scenarios, such as:
  4. Budgeted Single-agent: The model must allocate a fixed budget (e.g., €600) across six images.
  5. Valuator: A multi-agent setup where three model "profiles" bid on items, and the final score is the second-highest bid (simulating a Vickrey auction).
Figure 1: Conceptual progression from (a) LLM judgments evaluated against price-blind human rankings, to (b) prior price-assisted prediction with human preferences as targets, and (c) our test of transaction prices as candidate preference labels. Panel (c) measures price coverage and agreement with human rankings while preserving ties and conflicts. Prices are treated as a protocol-dependent proxy, not assumed to be ground truth.

Datasets

The study introduces MarketImageArt, which contains:

  • 1,596 artworks from emerging-art auctions (2020–2024).
  • 266 six-image sets partitioned into low, mid, and high-price strata.
  • 3,990 within-set image pairs with both realized transaction prices and independent, price-blind human rankings.

The prices in the dataset range from €10 to €395, focusing on a "near-tie" regime where visual differences should, in theory, be the primary driver of price.

Key Insights and Ideas

  • Price is a Sparse Signal: In the dataset, 57.9% of sets had all six prices tied. Only 24.2% of all image pairs had a clear price difference.
  • The Price-Human Gap: Even when a price difference existed, it only agreed with the human majority 34.7% of the time. In many cases, the market and the human eye were in direct conflict.
  • Order Bias is Real: The study found a significant "earlier-item" bias. Humans favored the earlier-positioned item in 65.9% of pairs. Because the dataset was sorted by price (lowest to highest), this created a massive confound where the "market" favored the later items while humans favored the earlier ones.
  • Mechanism Harm: One might assume that making a model act like a strategic bidder would improve its valuation accuracy. Surprisingly, for the Qwen model family, adding budget constraints and multi-agent "auctions" significantly lowered performance compared to simple direct scoring.
Figure 3: Conceptual comparison between direct visual judg- ment and budget-constrained multi-agent valuation. The implementations evaluated in Section 3.2 are elicitation in- terfaces, not reproductions of the original auctions.

Key Results

  • Recoverable Accuracy: On the subset where prices and humans actually agreed (the "protocol-supported" set), HPSv2 (a static visual preference model) performed best at 66.4% accuracy.
  • MLM Performance: GPT-5.6 Sol Pair reached 62.1%, while Qwen3-VL-4B Pair struggled significantly at only 8.4%, largely due to its extreme sensitivity to the order of images.
  • Elicitation Effects: For GPT models, complex interfaces (like the Valuator) showed slight point gains, but these were not statistically significant. For Qwen, the Valuator interface was "the most harmful tested contrast," dropping accuracy by 26.6 points compared to a single-agent baseline.

Limitations and Future Work

The authors acknowledge several limitations:

  • Scope: The data comes from a single, low-price emerging art source. Results might differ in high-end markets (e.g., Sotheby's) where provenance and investment value dominate even more.
  • Artificial Sets: The six-image sets were constructed for the experiment, not taken from simultaneous real-world auctions.
  • Order Confound: While they audited for order bias, the fixed-order nature of the human annotations remains a significant hurdle for establishing a clean "visual-only" signal.

Future Work: The authors suggest testing broader markets with randomized presentation orders to further untangle the complex relationship between what we see and what we are willing to pay. Not discussed in the paper is the potential for training models specifically to ignore market noise, which this diagnostic framework could facilitate.

Summary updated

Abstract

Existing visual preference evaluation relies on costly human annotations or preference reward models, whereas online marketplace transaction prices are readily available and reflect real purchasing behavior, offering potential low-cost supervision. However, transaction prices conflate visual preferences with non-visual factors such as provenance, demand, and competition, making aggregate accuracy insufficient to distinguish model errors, unreliable price signals, and protocol confounds. To address this identifiability challenge, we introduce MarketImage, a diagnostic framework that identifies when transaction prices reflect visual preferences, where they fail, and whether multimodal language models can recover human-supported visual preference signals. MarketImage comprises three components: (i) MarketImageArt, 266 six-image comparison sets with transaction prices and price-blind human rankings; (ii) MarketImageLens, a tie-preserving protocol that partitions 3,990 within-set image pairs into price-human agreement, conflict, and price-tie regions while separately auditing presentation-order effects; and (iii) MarketValuator, six elicitation methods spanning direct comparison, independent scoring, budget constraints, and multi-agent aggregation to evaluate signal recovery by static visual scorers and three multimodal language models. Experiments show that only 24.2% of image pairs have a non-tied price direction, of which 34.7% agree with the human majority. Within this region, HPSv2 leads at 66.4% Recoverable Accuracy; Qwen's budgeted and aggregated variants trail baselines, while GPT interface gains remain statistically unresolved. Thus, transaction prices encode recoverable visual preferences only under limited conditions and should be treated as uncertain proxies rather than ground truth.
BibTeX
@misc{ji2026do,
  author = {Fengxian Ji and Jingpu Yang and Fan Zhang and Junhong Liang and Qian Jiang and Yuke Li and Jinri Guo and Rongxin Zhu and Zhuohan Xie},
  title = {{Do Transaction Prices Reflect Visual Preference? A Diagnostic Study with Multimodal Language Models}},
  year = {2026},
  eprint = {2610.41358},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.41358},
}
Figure