Parcel loss prediction in last-mile delivery: deep and non-deep approaches with insights from Explainable AI

Jan de Leeuw, Zaharah Allah Bukhsh, Yingqian ZhangarXiv 2023 · DOI · arXiv

Introduction

Last-mile delivery represents the most volatile and expensive segment of the supply chain, often accounting for over half of total shipping costs. Within this phase, parcel loss is a critical issue that erodes customer trust and causes significant financial leakage. Currently, many e-commerce retailers rely on rudimentary, manual business rules to manage insurance risks. For example, a common rule might be to insure any shipment with a value exceeding a specific threshold, such as €250.

Figure 1: Example of simplistic manual business rules used for determining insurance value based on stock price.

The primary challenge in automating this process using Machine Learning is the extreme class imbalance; in typical logistics data, lost parcels constitute less than 0.5% of total shipments. This "needle-in-a-haystack" problem often causes standard algorithms to overlook losses entirely.

Methodology

This research proposes and compares two distinct frameworks to address the imbalance and improve prediction accuracy:

  1. DBSL (Data Balance with Supervised Learning): This approach utilizes traditional resampling techniques, such as Random Undersampling and UnderBagging, to balance the dataset before training standard classifiers like XGBoost or Random Forests.
  2. DHEL (Deep Hybrid Ensemble Learning): A more advanced approach that treats parcel loss as an anomaly detection task. A Deep Autoencoder (AE) is trained exclusively on successful deliveries to learn their "normal" reconstruction. When the AE encounters a lost parcel, the resulting high reconstruction error serves as a powerful feature for a secondary ensemble classifier.

Data and Feature Insights

The study analyzed a year of shipment data from a major Dutch retailer, comprising approximately 855,000 rows with a 0.25% loss rate. Using Explainable AI (XAI) techniques like SHAP values, several key risk factors were identified:

  • Product Type: High-value electronics (e.g., Apple products) were strong predictors of loss, suggesting theft or fraud.
  • Delivery Infrastructure: Contrary to expectations, parcel lockers showed higher loss rates in the 2020 dataset, likely due to the relative immaturity of the technology at the time.
  • Physical Dimensions: Both extremely small and extremely large parcels exhibited higher risk profiles compared to medium-sized items.
Figure 3: Boxplot comparing the length of parcels that were lost versus those successfully delivered.

Results and Impact

The DHEL model (AE-RF hybrid) outperformed traditional methods, achieving a Balanced Accuracy of 0.701 and a ROC-AUC of 0.763. Crucially, it detected 55.4% of lost parcels, a significant improvement over the 42.2% detection rate of existing business rules. Implementing these models could save the case study company over €550,000 annually in insurance premiums by enabling more targeted risk management.

Conclusion

While the DHEL model offers superior performance, it presents a trade-off in interpretability compared to simpler supervised models. Future improvements could involve incorporating customer behavioral history and continuously retraining the models to adapt to evolving fraud patterns and maturing logistics technologies.

Summary updated

Abstract

Within the domain of e-commerce retail, an important objective is the reduction of parcel loss during the last-mile delivery phase. The ever-increasing availability of data, including product, customer, and order information, has made it possible for the application of machine learning in parcel loss prediction. However, a significant challenge arises from the inherent imbalance in the data, i.e., only a very low percentage of parcels are lost. In this paper, we propose two machine learning approaches, namely, Data Balance with Supervised Learning (DBSL) and Deep Hybrid Ensemble Learning (DHEL), to accurately predict parcel loss. The practical implication of such predictions is their value in aiding e-commerce retailers in optimizing insurance-related decision-making policies. We conduct a comprehensive evaluation of the proposed machine learning models using one year data from Belgian shipments. The findings show that the DHEL model, which combines a feed-forward autoencoder with a random forest, achieves the highest classification performance. Furthermore, we use the techniques from Explainable AI (XAI) to illustrate how prediction models can be used in enhancing business processes and augmenting the overall value proposition for e-commerce retailers in the last mile delivery.
BibTeX
@misc{leeuw2023parcel,
  author = {Jan de Leeuw and Zaharah Allah Bukhsh and Yingqian Zhang},
  title = {{Parcel loss prediction in last-mile delivery: deep and non-deep approaches with insights from Explainable AI}},
  year = {2023},
  eprint = {2310.16602},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.16602},
}
Figure