A New Feature Based Deep Attention Sales Forecasting Model for Enterprise Sustainable Development

Huang Jian, Qinyu Chen, Chengqing YuSustainability 2022 · DOI

TL;DR

This paper introduces a novel sales forecasting model for e-commerce that combines Temporal Convolutional Networks (TCNs) for time-series feature extraction, reinforcement learning (Q-learning) for intelligent feature selection, and the Reformer architecture for its attention mechanism to predict sales with improved accuracy and stability. This approach aims to enhance the sustainable development of e-commerce enterprises by providing more reliable sales predictions.

What Problem Does This Paper Address?

E-commerce sales are notoriously complex and influenced by a multitude of factors, making accurate forecasting a significant challenge. Misjudging supply and demand can lead to substantial financial losses for businesses. While various forecasting methods exist, they often struggle with the high dimensionality of e-commerce data, the intricate correlations between features, and the need for timely adaptation to market dynamics. This paper addresses the need for a more robust and accurate sales forecasting model that can effectively handle these complexities, thereby supporting the sustainable growth of e-commerce businesses.

Methods

The proposed model, TCN-Q-Reformer, is a hybrid deep learning architecture with three key components:

  • Temporal Convolutional Network (TCN): This layer is responsible for extracting deep temporal features from historical sales data. TCNs are chosen for their ability to capture long-range dependencies in time-series data without the vanishing gradient problem often associated with Recurrent Neural Networks (RNNs).
  • Reinforcement Learning (Q-learning) for Feature Selection: To address the issue of high-dimensional and potentially redundant features, a Q-learning algorithm is employed. This module acts as an independent agent that learns to select the most relevant correlation features (e.g., average price, average freight, sales by category) that contribute most to accurate sales prediction. This process not only reduces dimensionality but also enhances the quality of input data.
  • Reformer Model: The final layer utilizes the Reformer architecture, a variant of the Transformer model designed for efficiency with long sequences. The Reformer's attention mechanism allows it to weigh the importance of different features (both temporal and selected correlation features) dynamically, leading to more stable and accurate sales forecasts.

Datasets

The study utilizes the Olist dataset, which contains information on approximately 100,000 orders from various markets in Brazil between 2016 and 2018. This dataset includes features such as order status, goods size, price, payment, merchant and customer locations, and customer reviews. For the purpose of this study, 10 transaction features and 74 sales characteristics by commodity category were extracted. The data was preprocessed into daily time series, resulting in a total of 500 data points. The first 80% of this data was used for training, and the remaining 20% for testing.

Key Insights and Ideas

  • The Power of Hybridization: The paper strongly advocates for combining different specialized neural network architectures and techniques. The synergy between TCN's temporal feature extraction, Q-learning's intelligent feature selection, and Reformer's attention mechanism is crucial for achieving superior performance.
  • Feature Engineering is Not Dead: Despite the rise of end-to-end deep learning, this work demonstrates that thoughtful feature engineering (in this case, selecting relevant correlation features) can significantly boost prediction accuracy, even for advanced models like Reformer.
  • Reinforcement Learning for Optimal Feature Selection: Using RL for feature selection is highlighted as a more efficient and intelligent approach compared to traditional meta-heuristic methods, allowing the model to adaptively find the best feature subsets.
  • Reformer's Attention for Stability: The Reformer's attention mechanism is shown to be particularly effective in stabilizing sales forecasts by focusing on the most impactful features, a critical aspect for enterprise planning.

Key Results

The proposed TCN-Q-Reformer model consistently outperformed various benchmark models, including traditional neural networks (TCN, GRU, LSTM, RNN, ELM, RBF) and other advanced deep learning frameworks.

  • Accuracy Improvements: The paper reports significant improvements in prediction accuracy across multiple metrics. For instance, in comparative experiments, the TCN-Q-Reformer achieved the lowest MAE, MAPE, and RMSE values compared to models like TCN-PSO-Reformer, TCN-GA-Reformer, and the standalone Reformer.
  • Quantitative Gains: Specific promoting percentages (PMAE, PMAPE, PRMSE) show that TCN-Q-Reformer achieved substantial improvements over single models. For example, it showed a 40.7730% improvement in PMAE and a 61.1702% improvement in PMAPE compared to the Reformer model alone.
  • Robustness: The model demonstrated higher stability and accuracy, as evidenced by its consistent performance across different experimental setups and its superior results compared to established models like MLR, SVM, Li's model, and Dong's model.

Limitations and Future Work

  • Single Predictor Limitations: The authors acknowledge that a single predictor model, while effective, can have limitations in adapting to highly complex and variable datasets.
  • Ensemble Learning: To further enhance model stability and adaptability, the authors suggest exploring ensemble learning models in future work.
  • Incorporating More Factors: The paper notes that sales are influenced by a broader range of factors beyond those considered, such as competition, social policies, and regional economic development. Future research could incorporate these additional factors for more comprehensive modeling.
  • Strategic Policy Formulation: The authors envision that the sales forecast results can provide valuable technical support for enterprises to develop future strategies and formulate key policies.

Summary updated

Abstract

In recent years, with the rise of the Internet, e-commerce has become an important field of commodity sales. However, e-commerce is affected by many factors, and the wrong judgment of supply and marketing relationships will bring huge losses to operators. Therefore, it is of great significance to establish a model that can effectively achieve high precision sales prediction for ensuring the sustainable development of e-commerce enterprises. In this paper, we propose an e-commerce sales forecasting model that considers the features of many aspects of correlation. In the first layer of the model, the temporal convolutional network (TCN) is used to extract the deep temporal characteristics of univariate sales historical data, which ensures the integrity of temporal information of sales characteristics. In the second layer, the feature selection method based on reinforcement learning is used to filter the effective correlation feature set and combine it with the temporal feature after processing, which not only improves the amount of effective information input by the model, but also avoids the high feature dimension. The third layer of the reformer model learns all the features and pays different attention to the features with different degrees of importance, ensuring the stability of the sales forecast. In the experimental part, we compare the proposed model with the current advanced sales forecasting model, and we can find that the proposed model has higher stability and accuracy.
BibTeX
@inproceedings{jian2022new,
  author = {Huang Jian and Qinyu Chen and Chengqing Yu},
  title = {{A New Feature Based Deep Attention Sales Forecasting Model for Enterprise Sustainable Development}},
  booktitle = {Sustainability},
  year = {2022},
  doi = {10.3390/su141912224},
}
Figure