IncVNM: An Interactive Recommender System with Incremental von Neumann-Morgenstern Utility Learning

Jiaxin Wei, Jia LiuarXiv 2026 · arXiv

TL;DR

IncVNM is a new interactive recommender system designed for products with uncertain outcomes (like stocks or insurance) that learns a user’s personal risk attitude—their von Neumann–Morgenstern (VNM) utility function—from simple click behavior. By updating this utility function incrementally and using it to evaluate probability distributions rather than static features, the system achieves superior accuracy and generalizes instantly to new products.

What Problem Does This Paper Address?

Most recommender systems are great at finding statistical associations: "Users who liked Inception also liked Interstellar." However, this approach breaks down when products have random outcomes, such as financial assets, insurance contracts, or services with volatile costs.

In these cases, the "value" of a product isn't a fixed set of features; it’s a probability distribution of potential outcomes. Furthermore, how a user values that distribution depends entirely on their individual risk attitude. A conservative investor and a high-stakes gambler might look at the exact same stock and reach opposite conclusions.

Traditional systems struggle here because:

  1. They don't account for the full distribution of outcomes.
  2. They fail to capture the underlying "utility" that drives a user's risk-averse or risk-seeking behavior.
  3. They lack transferability; if a stock's return distribution changes next month, the system often has to "re-learn" the user's preference for that specific item from scratch.

Methods

The authors introduce IncVNM, which treats recommendation as a continuous loop of preference elicitation and product-list generation.

The high-level interaction loop of the IncVNM system.

1. Turning Clicks into Comparisons

When a user browses a list of products and clicks on a few, the order of those clicks reveals their preferences. IncVNM interprets these "click sequences" as a set of pairwise comparisons between outcome distributions. For example, if you click Asset A before Asset B, the system assumes you prefer the distribution of A over B.

Figure 2: Pairwise comparisons from a click sequence.

2. Incremental Utility Learning

Instead of re-training a massive model, IncVNM performs a "lightweight" update. It maintains a piecewise-linear approximation of the user's VNM utility function. After every interaction, it solves a mathematical optimization problem (specifically, a second-order cone program) to find a new utility function that:

  • Satisfies the new pairwise preferences.
  • Stays as close as possible to the previous estimate (minimizing the Kantorovich distance).

3. Balancing Exploration and Exploitation

To decide what to show the user next, the system uses an entropy-regularized sampling policy. It uses a "softmax" distribution to favor items with high predicted utility (exploitation) while occasionally showing diverse items to learn more about the user's preferences (exploration).

4. Scalability via Finite Precision

To prevent the model from becoming too complex as more data arrives, the authors use a "finite-precision" representation. By rounding outcomes to a specific grid (e.g., rounding stock returns to the nearest 0.1%), they bound the number of "breakpoints" in the utility function, ensuring the system remains fast regardless of how large the product catalog grows.

Datasets

The researchers evaluated IncVNM using a stock recommendation simulation:

  • Data Source: Monthly returns for 463 constituent stocks of the S&P 500 (Jan 2022 – April 2026).
  • Protocol: A rolling-window experiment where the system "sees" 24 months of historical data to estimate return distributions, then simulates 20 user interactions before shifting the window forward by one month.
  • User Simulation: They simulated a risk-averse user and introduced "response errors" (accidental clicks or overlooked items) to test the system's robustness.

Key Insights and Ideas

  • Utility as a Transferable Representation: Because the system learns a utility function over outcomes (e.g., "how much do I like a 5% return?") rather than items, it can immediately evaluate a brand-new product it has never seen before, provided the product's outcome distribution is known.
  • The Power of Ordinal Feedback: You don't need users to fill out complex risk-assessment surveys. Simple browsing behavior—the order in which they click—is sufficient to reconstruct their internal utility curve.
  • Robustness to Noise: By using a "confidence-weighted" relaxation in their optimization, the system doesn't break when a user makes a mistake. It recognizes which comparisons are "stronger" based on the current model and penalizes violations of those more heavily.

Key Results

IncVNM was compared against several baselines, including standard financial metrics (Expected Return, Mean-Variance) and state-of-the-art recommendation algorithms (Discounted-UCB, Distribution-aware BPR).

  • Superior Accuracy: IncVNM substantially outperformed all baselines in ranking accuracy. In the error-free case, its "Ranking Difference" (a measure of how far the recommendations are from the user's true ideal) was nearly 20x lower than the next best learning-based method.
  • Resilience to Error: Even when 50% of the user's click sequences were perturbed with errors, IncVNM maintained high performance, while other feedback-driven methods saw significant degradation.
  • Efficiency: The update time remained stable over hundreds of interactions, confirming that the finite-precision approach successfully manages computational overhead.

Limitations and Future Work

While the results are promising, the authors acknowledge a few areas for growth:

  • Simulated Users: The current study relies on simulated click behavior based on mathematical utility models. Real-world human behavior is often more "irrational" or inconsistent than these models suggest.
  • Fixed Parameters: Some parameters, like the Lipschitz constant (which controls how "steep" the utility function can be), are currently fixed and might need to be learned or tuned for different domains.
  • Real-world Testing: The next logical step is evaluating IncVNM in live online experiments with actual users to see how it handles the complexities of real-world browsing.

Summary updated

Abstract

In modern recommender systems, users are often presented with products whose outcomes are random. The values of such products depend on their entire outcome distributions and on users' attitudes toward risk, making them difficult to evaluate using traditional product-level recommendation methods. We propose \textsc{IncVNM}, an interactive recommender system that incrementally learns a user-specific von Neumann--Morgenstern (VNM) utility function from ordinal click feedback generated during ordinary webpage browsing. At each interaction, \textsc{IncVNM} converts the click order within a displayed product list into pairwise comparisons between outcome distributions and updates the nominal utility by minimizing its deviation from the previous estimate while incorporating the newly observed pairwise comparisons. To close the interaction loop, an entropy-regularized sampling policy balances exploitation of the learned utility against exploration for additional preference information. We evaluate \textsc{IncVNM} in a rolling-window experiment with different response-error probabilities and show that it substantially outperforms the expected-return, mean--variance, discounted-UCB, and distribution-aware BPR baselines.
BibTeX
@misc{wei2026incvnm,
  author = {Jiaxin Wei and Jia Liu},
  title = {{IncVNM: An Interactive Recommender System with Incremental von Neumann-Morgenstern Utility Learning}},
  year = {2026},
  eprint = {2611.01786},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.01786},
}
Figure