Learning and Trust in Auction Markets

Pooya Jalaly, Denis Nekipelov, Éva TardosarXiv 2017 · arXiv

Introduction

Classical auction theory typically assumes participants are "Econs"—rational actors capable of instantaneous, optimal strategy calculations. However, real-world online advertising auctions, particularly those serving local markets like real estate, are often populated by individual human bidders. These agents frequently lack sophisticated analytical tools and are subject to behavioral biases. This research investigates the friction between platform-provided bid recommendations and human adoption, specifically within Zillow’s transition to an auction-based advertising system.

The Auction Environment

The study focuses on Zillow’s "Premier Agent" program, where agents bid for impressions on property detail pages. These impressions translate into lead generation opportunities when potential buyers use the "Contact Agent" feature.

A Zillow property detail page showing the 'Contact Agent' section, which serves as the primary inventory for the advertising auction.

When Zillow moved to an auction model, it introduced a recommendation tool to suggest optimal bids based on an agent's budget and desired visibility. Despite the tool's intent to simplify the process, many agents initially resisted its suggestions.

Methodology

To analyze bidder behavior without knowing their internal valuations for leads, the researchers employed a value inference methodology based on "rationalizable sets." By observing bidding histories, they identified the range of values that would make an agent's actions consistent with low-regret learning.

Furthermore, the team developed an auction simulator to account for Zillow's "budget smoothing" or pacing mechanisms. This allowed for a counterfactual analysis: comparing actual agent performance against what would have been achieved had they strictly followed the platform's recommendations. Bidders were also segmented using K-means clustering based on their bid-change frequency to proxy for the "cost" of active participation.

The Trust Gap and Experimentation

A primary finding of the research is the existence of a significant "trust gap." Upon entering the auction market, agents followed the recommendation tool less than 50% of the time. It took approximately five months for adherence to approach 100%.

This delay suggests that bidders view platform recommendations with skepticism, suspecting that the "black box" might prioritize platform revenue over bidder utility. Consequently, agents engage in costly "experimentation"—deviating from recommendations not necessarily to find a superior strategy, but to verify the fairness and efficacy of the tool through personal experience.

Key Results

The analysis of agent performance revealed a mixed landscape:

  • Performance Variance: Approximately 20.8% of agents performed worse than the recommendation tool, while 29.2% managed to outperform it. The remaining 50% performed equally, largely due to high adherence.
  • Adoption Dynamics: There was a clear correlation between effort and adoption. Bidders who adjusted their bids most frequently were the fastest to adopt the recommendation tool, suggesting that active learning leads to a quicker realization of the tool's optimality.
  • Market Maturity: While "irrational" experimentation decreased as markets matured, it remained a persistent factor, indicating that human distrust is a structural element of platform design that does not entirely disappear with time.

Conclusion and Implications

The study highlights that for platforms involving individual human bidders, technical optimality is not enough; trust is a prerequisite for efficiency. The "trust tax" paid by skeptical bidders—in the form of lost utility during experimentation—suggests that platforms might benefit from more transparent bidding mechanisms or simplified interfaces. For markets dominated by non-algorithmic participants, moving toward automated optimization by default may protect users from the financial consequences of their own skepticism and lack of expertise.

Summary updated

Abstract

In this paper, we study behavior of bidders in an experimental launch of a new advertising auction platform by Zillow, as Zillow switched from negotiated contracts to using auctions in several geographically isolated markets. A unique feature of this experiment is that the bidders in this market are real estate agents that bid on their own behalf, not using third-party intermediaries. To help bidders, Zillow also provided a recommendation tool that suggested a bid for each bidder. Our main focus in this paper is on the decisions of bidders whether or not to adopt the platform-provided bid recommendation. We observe that a significant proportion of bidders do not use the recommended bid. Using the bid history of the agents we infer their value, and compare the agents' regret with their actual bidding history with results they would have obtained following the recommendation. We find that for half of the agents not following the recommendation, the increased effort of experimenting with alternate bids results in increased regret, i.e., they get decreased net value out of the system. The proportion of agents not following the recommendation slowly declines as markets mature, but it remains large in most markets that we observe. We argue that the main reason for this phenomenon is the lack of trust in the platform-provided tool. Our work provides an empirical insight into possible design choices for auction-based online advertising platforms. While search advertising platforms (such as Google or Bing) allow bidders to submit bids on their own, many display advertising platforms (such as Facebook) optimize bids on bidders' behalf and eliminate the need for bids. Our empirical analysis shows that the latter approach is preferred for markets where bidders are individuals, who don't have access to third party tools, and who may question the fairness of platform-provided suggestions.
BibTeX
@misc{jalaly2017learning,
  author = {Pooya Jalaly and Denis Nekipelov and Éva Tardos},
  title = {{Learning and Trust in Auction Markets}},
  year = {2017},
  eprint = {1703.10672},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1703.10672},
}
Figure