Neural Utility Functions

Porter Jenkins, Ahmad Farag, J. Stockton Jenkins, Huaxiu Yao, Suhang Wang, Zhenhui Li

Introduction

Modern recommendation engines are typically optimized for prediction accuracy, minimizing the error between predicted and actual ratings. However, these models are often "economically illiterate," failing to account for the fundamental trade-offs between substitutes and complements. This paper introduces Neural Utility Functions (NUFs), a framework that embeds economic inductive biases into neural networks to produce more rational and effective recommendations.

The Rationality Gap in Deep Learning

In microeconomics, consumer behavior is modeled through Utility Theory, which assumes users seek to maximize a "utility function." A critical property of these functions is quasi-concavity, representing diminishing marginal returns and logical trade-offs. The authors demonstrate that standard two-layer neural networks are mathematically "irrational" because they do not inherently satisfy quasi-concavity. In high-dimensional spaces, the probability of a standard network behaving like a valid utility function is nearly zero, leading to recommendations that ignore basic economic relationships.

Neural Utility Functions (NUF)

The proposed NUF approach is a backbone-agnostic framework that can be integrated into existing architectures like Matrix Factorization (MF) or Deep Factorization Machines (DFM).

The NUF framework integrates a utility-based loss term alongside standard prediction loss.

The core of the method is a specialized loss function inspired by the Marginal Rate of Substitution (MRS). By constraining the gradients of the model, NUFs force the network to learn smoother, more logical internal representations. The training process involves:

  1. The Utility Prior: A new loss term ($\mathcal{L}_{NUF}$) that aligns model gradients with economic principles.
  2. Gradient Constraints: Minimizing the gradient ratio between an item and its complements while maximizing it against substitutes.
  3. Sampling Strategy: For each user-item pair, the model samples complements (items the user liked) and substitutes (items the user did not choose) to calculate these constraints during training.

Experimental Evaluation

The framework was tested on both simulated and real-world datasets:

  • Simulated Data: Testing the recovery of theoretical substitution effects using Cobb-Douglas and CES functions.
  • MovieLens 25M: A benchmark for explicit movie ratings.
  • Amazon 18: Focused on Grocery and Home categories to evaluate real-world product relationships.

Results and Analysis

The inclusion of NUF loss consistently improved performance across four state-of-the-art architectures.

Performance comparison across multiple architectures and datasets shows consistent improvements with NUF.

Key findings include:

  • Accuracy Gains: NUF improved recommendation accuracy (measured by RMSE and NDCG) in 81% of experimental cases with statistical significance.
  • Theoretical Alignment: On simulated data, NUFs were significantly more accurate at identifying true substitution rates than vanilla neural networks.
  • Economic Analogies: The model successfully learned complex item relationships. For instance, it identified analogies such as Coffee - Creamer + Pizza ≈ Cheddar, demonstrating an understanding of how complements shift across different contexts (e.g., swapping a breakfast complement for a dinner complement).

Conclusion

By bridging the gap between classical economic theory and modern deep learning, Neural Utility Functions provide a path toward AI that understands the underlying logic of human choice. While the method introduces some computational overhead due to in-graph gradient calculations, the resulting gains in recommendation rationality and accuracy make it a compelling addition to the recommender systems toolkit.

Summary updated

Abstract

Current neural network architectures have no mechanism for explicitly reasoning about item trade-offs. Such trade-offs are important for popular tasks such as recommendation. The main idea of this work is to give neural networks inductive biases that are inspired by economic theories. To this end, we propose Neural Utility Functions, which directly optimize the gradients of a neural network so that they are more consistent with utility theory, a mathematical framework for modeling choice among items. We demonstrate that Neural Utility Functions can recover theoretical item relationships better than vanilla neural networks, analytically show existing neural networks are not quasi-concave and do not inherently reason about trade-offs, and that augmenting existing models with a utility loss function improves recommendation results. The Neural Utility Functions we propose are theoretically motivated, and yield strong empirical results.
BibTeX
@inproceedings{jenkins2021neural,
  author = {Porter Jenkins and Ahmad Farag and J. Stockton Jenkins and Huaxiu Yao and Suhang Wang and Zhenhui Li},
  title = {{Neural Utility Functions}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i9.16966},
}
Figure