Two-level monotonic multistage recommender systems

Ben Dai, Xiaotong Shen, Wei PanarXiv 2021 · DOI · OpenReview

Abstract

A recommender system learns to predict the user-specific preference or intention over many items simultaneously for all users, making personalized recommendations based on a relatively small number of observations. One central issue is how to leverage three-way interactions, referred to as user-item-stage dependencies on a monotonic chain of events, to enhance the prediction accuracy. A monotonic chain of events occurs, for instance, in an article sharing dataset, where a "follow" action implies a "like" action, which in turn implies a "view" action. In this article, we develop a multistage recommender system utilizing a two-level monotonic property characterizing a monotonic chain of events for personalized prediction. Particularly, we derive a large-margin classifier based on a nonnegative additive latent factor model in the presence of a high percentage of missing observations, particularly between stages, reducing the number of model parameters for personalized prediction while guaranteeing prediction consistency. On this ground, we derive a regularized cost function to learn user-specific behaviors at different stages, linking decision functions to numerical and categorical covariates to model user-item-stage interactions. Computationally, we derive an algorithm based on blockwise coordinate descent. Theoretically, we show that the two-level monotonic property enhances the accuracy of learning as compared to a standard method treating each stage individually and an ordinal method utilizing only one-level monotonicity. Finally, the proposed method compares favorably with existing methods in simulations and an article sharing dataset.

Introduction

In modern recommender systems, user behavior is rarely a single isolated event. Instead, it often follows a logical, sequential progression known as a monotonic behavior chain. For example, on a content platform, a user must "view" an article before they can "like" it, and they are unlikely to "follow" the author unless they have liked the content.

A monotonic behavior chain illustrating the progression from viewing to following.

Standard recommender systems often treat these stages independently, which leads to two primary issues: logical inconsistency (predicting a "follow" without a "like") and data sparsity (later stages have far fewer positive observations). The framework proposed by Dai, Shen, and Pan addresses these by enforcing "two-level monotonicity" to ensure logical integrity and improve predictive power by leveraging dependencies across the entire chain.

Methodology

The core of the proposed system is the Two-level Monotonic Property, which encompasses both forward and backward logical constraints:

  • Forward Monotonicity: If a user does not perform an action at a current stage, they cannot perform it at any subsequent stage.
  • Backward Monotonicity: If a user performs an action at a future stage, they must have performed all preceding actions in the chain.

To mathematically guarantee these properties, the authors introduce a nonnegative additive latent factor model. By representing the decision function as a base preference minus a series of nonnegative "costs" for reaching subsequent stages, the model is structurally incapable of producing inconsistent predictions.

The proposed model architecture integrating continuous and categorical features into user-item-stage interactions.

As shown in the architecture above, the model integrates continuous user/item features ($u_i, v_j$) and categorical features ($s_i, o_j$) into latent factors. These factors are then combined to model the user-item-stage interaction. The system utilizes a large-margin classifier (adapting hinge loss for multistage pairs) and a blockwise coordinate descent algorithm for efficient optimization.

Experimental Results

The model was evaluated using the Deskdrop dataset, a real-world article-sharing benchmark containing ~73,000 users and ~3,000 articles. The data follows a clear View $\rightarrow$ Like $\rightarrow$ Follow chain and is extremely sparse (less than 0.2% observed interactions).

Key findings include:

  • Superior Accuracy: The method outperformed SVD++, Gradient Boosting, and Deep Neural Networks, reducing error rates by 8.2% to 75.3%.
  • Guaranteed Consistency: While baseline models produced logically inconsistent predictions in up to 13.7% of cases, the proposed model maintained 0% inconsistency.
  • Resilience to Sparsity: By "borrowing strength" from denser early stages (like views), the model significantly improved performance in the data-scarce "Follow" stage.

Conclusion

By treating multistage interactions as a unified, monotonic process rather than independent events, this framework provides a robust solution for complex recommendation environments. While currently optimized for linear chains, the use of consistency as a regularizer offers a powerful template for future work in more complex, branching behavior trees and large-scale industrial applications.

Summary updated

Abstract

A recommender system learns to predict the user-specific preference or intention over many items simultaneously for all users, making personalized recommendations based on a relatively small number of observations. One central issue is how to leverage three-way interactions, referred to as user-item-stage dependencies on a monotonic chain of events, to enhance the prediction accuracy. A monotonic chain of events occurs, for instance, in an article sharing dataset, where a ``follow'' action implies a ``like'' action, which in turn implies a ``view'' action. In this article, we develop a multistage recommender system utilizing a two-level monotonic property characterizing a monotonic chain of events for personalized prediction. Particularly, we derive a large-margin classifier based on a nonnegative additive latent factor model in the presence of a high percentage of missing observations, particularly between stages, reducing the number of model parameters for personalized prediction while guaranteeing prediction consistency. On this ground, we derive a regularized cost function to learn user-specific behaviors at different stages, linking decision functions to numerical and categorical covariates to model user-item-stage interactions. Computationally, we derive an algorithm based on blockwise coordinate descent. Theoretically, we show that the two-level monotonic property enhances the accuracy of learning as compared to a standard method treating each stage individually and an ordinal method utilizing only one-level monotonicity. Finally, the proposed method compares favorably with existing methods in simulations and an article sharing dataset.
BibTeX
@misc{dai2021two,
  author = {Ben Dai and Xiaotong Shen and Wei Pan},
  title = {{Two-level monotonic multistage recommender systems}},
  year = {2021},
  eprint = {2110.06116},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.06116},
}
Figure