TL;DR
This paper introduces a novel meta-learning based stacked regression model for predicting Customer Lifetime Value (CLV). By combining the strengths of multiple base models (Random Forest, XGBoost, ElasticNet) and using a linear regressor as a meta-model, their approach achieves superior performance and interpretability compared to existing methods and deep learning models on a real-world e-commerce dataset.
What Problem Does This Paper Address?
Understanding Customer Lifetime Value (CLV) is crucial for businesses to effectively target high-value customers, expand revenue, and manage resources. CLV represents the total monetary value a customer is expected to bring to a business over their entire relationship. While various methods exist, traditional distribution-based and RFM (Recency, Frequency, Monetary) models struggle with a wide range of input features. Advanced deep learning models, though powerful, can introduce undesirable complexity and a "black box" nature, making them less interpretable. This paper aims to bridge this gap by proposing a model that is effective, comprehensive, simple, and interpretable for CLV prediction, particularly in non-contractual, continuous transaction environments.
Methods
The core of the proposed solution is a meta-learning based stacked regression approach. This method involves two levels:
- Level 1 (Base Models): Several well-performing regression models are trained independently on the input features. The paper utilizes a heterogeneous set of base models:
- Random Forest: A popular ensemble method known for its robustness.
- XGBoost: An optimized gradient boosting algorithm, recognized for its speed and performance.
- ElasticNet: A linear regression model that combines L1 and L2 regularization, offering a balance between feature selection and coefficient shrinkage.
- Level 2 (Meta-Model): The predictions generated by the Level 1 models are then used as input features for a meta-model. In this work, a linear regressor (specifically, ElasticNet) is employed as the meta-model. This "generalizer" learns how to best combine the predictions from the base models to produce the final CLV prediction.
To mitigate overfitting and improve generalization, the authors employ StackingCVRegressor, which leverages out-of-fold predictions. This involves splitting the data into k-folds, training the base models on k-1 folds, and using the remaining fold for predictions. These out-of-fold predictions are then stacked and fed to the meta-regressor.
Datasets
The paper uses the 'Online Retail II' dataset made available by UCI. This dataset comprises transactions from a UK-based, registered, non-store online retailer, covering the period from December 1, 2009, to December 30, 2011. The dataset includes attributes such as Invoice, StockCode, Description, Quantity, InvoiceDate, Price, CustomerID, and Country.
Key Insights and Ideas
- Stacking for CLV is a viable and effective approach: The paper demonstrates that combining diverse base models through stacking can lead to improved CLV prediction accuracy and robustness.
- Heterogeneity is key in base models: Using models with different underlying mechanisms (tree-based ensembles like Random Forest and XGBoost, and linear models like ElasticNet) allows the ensemble to capture a wider range of patterns in the data.
- Interpretability can be maintained with stacking: While deep learning models can be black boxes, the proposed stacked approach, using a linear meta-model and analyzing feature importance from base models, offers a better balance between performance and interpretability.
- Feature importance varies across models: The study highlights that different models (e.g., XGBoost) identify different features as most important for CLV prediction (e.g.,
freq_3mandlatetime), underscoring the benefit of using multiple models.
Key Results
The proposed stacked regression model achieved competitive results on the 'Online Retail II' dataset:
- Outperformed baseline BG/NBD: The stacked model significantly outperformed the traditional BG/NBD model in both RMSE and MAE.
- Competitive with other ML models: The stacked model achieved the lowest MAE (0.82) and the second-lowest RMSE (1.37), closely matching the performance of XGBoost (RMSE: 1.34, MAE: 0.83).
- Better than DNN: The stacked regressor model performed better than a tuned DNN on this specific dataset.
- Feature Importance: For XGBoost,
freq_3mandlatetimewere identified as the most important features based on 'Gain' and 'Weight' metrics.
Limitations and Future Work
- Limited Feature Set: The authors acknowledge that the dataset lacks other potentially informative features such as customer age, behavioral data, or marketing promotion information, which could further improve CLV prediction.
- Temporal Patterns: Incorporating temporal patterns like seasonality and trends using encoder-decoder RNNs is suggested for future work.
- Embeddings for Feature Engineering: Using embeddings for data pre-processing and feature engineering is proposed as a promising avenue.
- Broader Application: Testing the proposed model in different contexts or for other aspects of CLV prediction (e.g., churn prediction) would help assess its versatility.
- Interpretability Enhancement: While the model offers better interpretability than deep learning, further exploration of methods like LIME could enhance this aspect.
Summary updated