Abstract
Recent progress in Next-basket Recommendation (NBR) has relied heavily on Personalized Item Frequency (PIF). However, this creates a "frequency bias," where systems over-recommend items a user has purchased frequently in the past, narrowing their browsing scope and reducing system utility. We propose FENDER (Frequency-aware Deconfounder for Next-basket Recommendation), a causal inference-based model that treats user and item representations as unobserved confounders. By decomposing PIF into substitute confounders, FENDER provides more diverse and accurate recommendations. Experimental results across three large datasets demonstrate that FENDER outperforms ten baseline models in both accuracy and diversity.
Introduction
In the retail sector, Next-Basket Recommendation (NBR) systems aim to predict the set of items a user will purchase in their next transaction. While effective, many current models suffer from a "filter bubble" effect. By over-relying on Personalized Item Frequency (PIF)—the historical frequency of a user's interactions with an item—these systems become biased toward frequent purchases. This "frequency bias" prevents the discovery of new interests and reduces the overall utility of the recommendation system.
The Causal Inference Problem
The authors frame frequency bias as a causal inference challenge. In a standard recommendation setting, unobserved factors such as a user's underlying personality or an item's intrinsic appeal act as confounders. These factors influence both the historical purchase frequency ($P_t$) and the future basket composition ($R_t$).
Without accounting for these hidden variables, models erroneously assume that past frequency is the direct cause of future behavior, leading to a self-reinforcing loop of repetitive recommendations. This feedback loop trap means models aren't just reflecting reality; they are amplifying existing biases.
Methodology: The FENDER Framework
To address this, the researchers developed FENDER (Frequency-aware Deconfounder for Next-basket Recommendation). The model employs a two-stage deconfounding process:
- PIF Decomposition: A Neural Tensor Layer (NTL) factorizes historical PIF data into latent user and item embeddings, capturing long-term preference patterns.
- Next-Basket Prediction: The model learns separate embeddings to represent unobserved confounders (current interests).
- Deconfounding Inference: By combining these signals with a tunable weight parameter ($\omega$), FENDER can balance conservative (habit-based) and diverse (interest-based) recommendations.
Experimental Evaluation
FENDER was tested on three major datasets: Instacart (3M+ orders), Dunnhumby (in-store transactions), and a proprietary Walmart Grocery dataset. The study focused on users with a history of at least five baskets to properly evaluate long-term bias. To measure success, the authors introduced the negative Top-Frequency Ratio (nTFR), which specifically tracks how many non-frequent items successfully reach the top of the recommendation list.
Results and Performance
FENDER consistently outperformed ten baseline models, including MIT-GNN and Sets2Sets. It achieved an average improvement of 10.20% in NDCG@20, demonstrating that mitigating bias does not have to come at the cost of accuracy.
Furthermore, FENDER showed significant improvements in diversity (nTFR) and demonstrated robustness against "noise" items in user history, correctly identifying them as irrelevant rather than frequent interests. This allows the system to break the "repeat purchase" cycle while maintaining high precision.
Conclusion
FENDER provides a robust framework for breaking the repetitive cycle of frequency-biased recommendations. By leveraging causal deconfounder theory, it balances historical habits with potential new interests. While the current model relies on pre-calculated PIF features, future work will explore integrating more complex sequential patterns, such as seasonality and "running out" cycles, into the deconfounding process.
Summary updated