Authors: Daron Acemoglu (MIT), Azarakhsh Malekian (University of Toronto), Ali Makhdoumi (Duke University), and Asuman Ozdaglar (MIT)
Abstract
We build a model of online behavioral manipulation driven by AI advances. A platform dynamically offers one of $n$ products to a user who slowly learns product quality. User learning depends on a product's "glossiness," which captures attributes that make products appear more attractive than they are. AI tools enable platforms to learn glossiness and engage in behavioral manipulation. We establish that AI benefits consumers when glossiness is short-lived. In contrast, when glossiness is long-lived, users suffer because of behavioral manipulation. Finally, as the number of products increases, the platform can intensify behavioral manipulation by presenting more low-quality, glossy products.
Introduction
In the digital age, the proliferation of data and the advancement of AI have sparked a debate between two distinct viewpoints. Economists and tech optimists argue that "big data" facilitates better product matching and more informative advertising, ultimately benefiting consumers by reducing search costs. Conversely, critics warn of "surveillance capitalism," where data is leveraged to modify human behavior for profit.
This paper seeks to reconcile these views by asking: Under what specific conditions does behavioral modification become behavioral manipulation? The authors define manipulation as a scenario where a platform uses its superior information to steer users toward low-quality products that appear attractive in the short term, thereby reducing overall consumer welfare.
The Model: Multi-Armed Bandits and Glossiness
The authors develop a theoretical model using a multi-armed bandit framework in continuous time. The user chooses between $n$ products, learning about their quality through consumption.
A key innovation in this model is the concept of glossiness ($\alpha$). Glossiness represents a temporary state where a low-quality product masks its true nature—for example, through a sleek interface or misleading marketing.
- Learning Dynamics: If a product is low-quality and not glossy, "bad news" (evidence of low quality) arrives at a rate $\gamma$.
- Persistence: Glossiness transitions to a non-glossy state at a rate $\rho$.
- Information Asymmetry: The paper compares a Pre-AI environment (where neither the platform nor the user knows the initial glossiness) to a Post-AI environment (where the platform uses aggregate data to perfectly estimate a product's initial glossiness).
Behavioral Bias: The Berk-Nash Equilibrium
A crucial assumption is that users are Bayesian but not "hyper-rational." They do not realize the platform has an informational edge and therefore do not adjust their beliefs based on the platform's specific product offers. This is modeled using the Berk-Nash equilibrium, capturing how users can be systematically misled when a platform possesses superior predictive power.
Key Insights: Helpfulness vs. Manipulation
The introduction of AI creates two opposing forces:
- The Helpfulness Effect: The platform guides users toward high-quality products faster than they would find them independently.
- The Manipulation Effect: The platform exploits its knowledge of "glossy" low-quality products to extract surplus before the user realizes the product's true quality.
The ultimate impact on welfare hinges on the persistence of glossiness ($\rho$).
- Welfare Gains (High $\rho$): When glossiness is transient, the platform cannot profit significantly from deception. The helpfulness effect dominates, increasing expected user utility and total welfare.
- Welfare Losses (Low $\rho$): When glossiness is persistent, the platform finds it more profitable to "trap" the user in a glossy, low-quality experience. In this case, the Post-AI environment leads to lower user utility and lower total welfare, despite increasing platform profits.
The "Double Whammy" of Choice
In standard economic models, more choice is generally better for the consumer. However, this model reveals that as the number of products ($n$) increases, the platform has more opportunities to find "glossy" low-quality items to present to the user. Consequently, for a sufficiently small transition rate $\rho$, increasing the number of products actually decreases expected user utility. Advances in AI that simultaneously increase product variety and predictive power can thus be twice as harmful to consumers.
Conclusion and Future Directions
The paper highlights that while AI can improve matching, it also enables sophisticated behavioral manipulation. The results suggest a need for regulatory frameworks to counter these effects, especially in environments where deceptive features are persistent. Future research could explore the impact of user "savviness," competition between platforms, and specific policy interventions to mitigate the harms of behavioral manipulation.
Summary updated