TAMPAR: Visual Tampering Detection for Parcel Logistics in Postal Supply Chains

Alexander Naumann, Felix Hertlein, Laura Dörr, Kai FurmansWACV 2024

TL;DR

This paper introduces TAMPAR, a novel dataset and pipeline for detecting visual tampering on parcels in postal supply chains using only a single RGB image. By leveraging keypoint detection to create viewpoint-invariant parcel surface representations and then applying change detection, they achieve promising results for identifying unauthorized modifications like new tape or labels.

What Problem Does This Paper Address?

The increasing volume of valuable goods transported through postal supply chains necessitates robust security measures. A critical aspect of this is ensuring the integrity of parcels, meaning detecting any signs of tampering that might indicate theft or unauthorized access. Traditional methods often rely on multiple cameras or controlled environments, which are not feasible for last-mile delivery scenarios where only a single, potentially low-quality image of a parcel is available. The challenge lies in reliably identifying a parcel and then detecting subtle appearance changes on its surfaces, despite variations in viewing angles, lighting, and potential parcel damage.

Methods

The proposed solution is a two-stage pipeline:

  1. Parcel Keypoint Detection: The authors use a Mask R-CNN with a keypoint head to detect the eight corner points of a parcel. A novel, unambiguous keypoint ordering is introduced to ensure consistency, especially for parcels that lack a clear "front" or "back." This keypoint information is crucial for the next step.
  2. Change Detection: Once the keypoints are identified, a perspective transformation is applied to create normalized, fronto-parallel views of the parcel's visible side surfaces. This effectively removes viewpoint variance. The problem then reduces to comparing these normalized views against a reference image from a database. The paper evaluates several image homogenization techniques (e.g., DexiNed, Canny) and similarity metrics (e.g., LPIPS, SSIM), with a particular focus on SimSaC, a recent approach that handles imperfect matches and geometric transformations robustly.

Datasets

  • Parcel3D: Used for training the keypoint detection model. This dataset contains synthetic images of cuboid-shaped and damaged parcels with 3D annotations.
  • TAMPAR (TAMpering detection dataset for PARcels): A newly introduced dataset specifically for this work. It consists of paired images of parcels (original and tampered) captured in a realistic logistics use-case scenario. It includes various tampering types (labels, tape, writing) with easy and hard variants. TAMPAR provides 888 training/validation images and 1842 test images for change detection.

Key Insights and Ideas

  • Keypoints as a Foundation for Viewpoint Invariance: The core idea is that accurately detecting parcel corners allows for the creation of normalized views, simplifying the complex problem of 3D object comparison in 2D images.
  • Unambiguous Keypoint Ordering: Developing a consistent ordering for parcel keypoints is essential for training deep learning models, as parcels lack the inherent front/back orientation of vehicles.
  • SimSaC's Robustness: The SimSaC method, combined with LPIPS as a similarity metric, emerges as the most effective approach for change detection, demonstrating its ability to handle real-world variations.
  • Domain Gap is Manageable: While training on synthetic data (Parcel3D) and evaluating on real data (TAMPAR, Parcel2D Real) shows a performance drop, the results are still promising, indicating the generalizability of the keypoint detection approach.

Key Results

  • Keypoint Detection: Achieved a Keypoint AP of 75.76 on the Parcel2D Real dataset and 97.18 on the TAMPAR dataset when trained solely on synthetic data.
  • Tampering Detection: The best performing system, combining SimSaC with LPIPS and using predicted keypoints, reached 0.81 accuracy and an F1-Score of 0.83 on the TAMPAR dataset.
  • Sensitivity Analysis: The approach shows robustness to lens distortion and viewing angles within a reasonable range. Tampering detection performance varies by type, with labels being easiest to detect and hard tape/writing being more challenging.

Limitations and Future Work

  • Limitations:
    • The current keypoint ordering is not invariant to horizontal flipping.
  • While the approach is robust to moderate lens distortion and viewing angles, extreme cases can degrade performance.
  • The sensitivity analysis for writing showed surprising performance degradation when using ground truth keypoints, possibly due to inaccurate enlargement of the region of interest.
  • Future Work:
    • Incorporate advanced keypoint detectors and explore shape priors (e.g., vanishing point loss, 2D/3D consistency).
  • Fine-tune SimSaC specifically for tampering detection.
  • Investigate more complex decision rules, such as random forests, for improved performance, especially with more diverse datasets.

Summary updated

BibTeX
@inproceedings{naumann2024tampar,
  author = {Alexander Naumann and Felix Hertlein and Laura Dörr and Kai Furmans},
  title = {{TAMPAR: Visual Tampering Detection for Parcel Logistics in Postal Supply Chains}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure