A Vision-Based Method for Detecting the Position of Stacked Goods in Automated Storage and Retrieval Systems

Chuanjun Chen, Junjie Liu, Haonan Yin, Biqing HuangSensors 2025 · DOI

TL;DR

This paper introduces STEGNet, a novel vision-based system for precisely detecting the 3D position of stacked goods in automated warehouses. By combining a powerful object detection network with geometric edge detection and a PnP algorithm, it achieves high accuracy and efficiency, even in challenging, real-world warehouse conditions. This work offers a robust solution for automated cargo monitoring, crucial for modern logistics.

What Problem Does This Paper Address?

Modern logistics heavily relies on Automated Storage and Retrieval Systems (AS/RS) for efficiency. However, ensuring the stability and safety of stacked goods within these systems is a significant challenge. Traditional manual inspections are too slow and error-prone for high-intensity operations. While existing automated vision systems exist, they often struggle to balance detection accuracy, computational speed, and adaptability to the complex and dynamic warehouse environment. These environments present issues like variable lighting, occlusions from background structures, and dynamic elements. Current vision approaches often fall short by either being computationally expensive (semantic segmentation) or lacking precise 3D spatial information (basic object detection). This paper aims to bridge this gap by developing a system that is accurate, efficient, and robust to real-world warehouse complexities.

Methods

The core of the proposed solution is a novel vision-based detection framework with three main components:

  1. Stack Surface Detection (STEGNet): This is a deep learning-based object detection network. Its key innovation is the Efficient Gated Pyramid Feature Network (EG-FPN), which integrates a Gated Feature Fusion (GFF) module and a Lightweight Attention Mechanism (LAM). This design optimizes feature extraction and fusion across different scales, improving accuracy while maintaining efficiency. STEGNet is built upon the STCNet architecture but enhances its neck and head components. A lightweight variant, MixFEGNet, is also proposed, using the MixFormer backbone for further efficiency gains.
  2. Box Edge Detection: This component uses a geometric constraint method. It involves several steps:
  3. Image Preprocessing: Grayscale conversion, contrast enhancement (CLAHE), Gaussian filtering, morphological operations, and median filtering to reduce noise and enhance features.
  4. Edge Detection: The Canny operator is used to extract edge contours.
  5. Edge Filtering: The Hough Transform (specifically Probabilistic Hough Transform - PHT) is employed to filter and classify edge points, identifying major lines and distinguishing true box edges from background noise.
  6. Line Fitting: The RANSAC algorithm is used for robust line fitting, effectively handling noisy edge points to determine accurate boundary line parameters.
  7. Corner Detection and Selection: A method based on line intersection computation and clustering is used to find precise corner coordinates, employing spatial geometric constraints and a multi-layer selection strategy.
  8. Pose Estimation: A Perspective-n-Point (PnP)-based 2D-to-3D transformation approach is used. This algorithm takes the detected 2D corner points, box dimensions, and camera intrinsic parameters to calculate the precise 3D coordinates and pose of each box. The paper also details camera calibration procedures and a multi-view extension for enhanced robustness.

Datasets

  • GY-WSBW-4D (Proposed): A custom multi-angle pallet image dataset constructed from a conveyor line in a medical industry automated warehouse. It contains 3024 photographs captured from four distinct angles, with detailed annotations for box dimensions, cargo names, specifications, and stacking patterns.
  • WSBW-Err (Proposed): A supplementary dataset of approximately 150 images specifically containing unstable stacking patterns, used for position detection algorithm experiments.
  • WSGID-B: An existing dataset used for comparative experiments.

Key Insights and Ideas

  • Synergy of Deep Learning and Geometry: The paper demonstrates that combining powerful deep learning-based object detection (STEGNet) with geometric edge detection and PnP for pose estimation is highly effective for 3D object localization in complex environments. This hybrid approach leverages the strengths of both paradigms.
  • EG-FPN for Efficient Feature Fusion: The novel EG-FPN module, with its Gated Feature Fusion and Lightweight Attention Mechanism, is crucial for optimizing multi-scale feature extraction and fusion. This is a key enabler for achieving high accuracy with reduced computational cost.
  • RANSAC for Robust Line Fitting: Using RANSAC for line fitting in the edge detection pipeline significantly improves robustness against noisy edge points and outliers, leading to more accurate boundary detection.
  • Lightweight Variant for Practical Deployment: The development of MixFEGNet shows that a significant reduction in model size and increase in inference speed can be achieved with minimal accuracy loss, making the solution more practical for resource-constrained warehouse environments.
  • Addressing Real-World Warehouse Challenges: The research explicitly tackles issues like variable lighting, background clutter, and challenging stacking patterns, which are often overlooked in idealized lab settings.

Key Results

  • STEGNet Performance: Achieved 93.49% mAP on the proposed GY-WSBW-4D dataset and 83.2% mAP on the WSGID-B dataset, outperforming existing benchmarks.
  • MixFEGNet Efficiency: The lightweight variant reduced model size by 34% and increased inference speed by 68% while maintaining competitive accuracy.
  • Pose Estimation Accuracy: Achieved Mean Absolute Error within 4 cm and Rotation Angle Error below 2° in practical applications.
  • WSBW-Err Dataset Performance: STEGNet showed superior performance on this dataset for unstable stacking patterns, with MAE of 3.79 cm, RMSE of 5.88 cm, RAE of 1.5°, 63.27% PA, and 80.8% BIA.
  • Ablation Studies: Demonstrated the effectiveness of EG-FPN components (Auto Fusion, LSA, ECA) and the Dynamic Head, showing significant performance improvements.

Limitations and Future Work

  • Color Dependency: The algorithm is primarily optimized for natural-color cardboard boxes. Performance shows variations with different colored boxes, especially low-contrast dark boxes, where edge detection accuracy decreases.
  • Lighting Sensitivity: Performance varies under different lighting conditions, with edge detection accuracy decreasing significantly under dim and uneven lighting.
  • Environmental Constraints: Background clutter significantly impacts performance, requiring additional preprocessing.
  • Geometric Limitations: The current implementation struggles with very high cargo stacks and cases where box edges adhere closely, limiting its application in densely packed scenarios.
  • Parameter Sensitivity: The method requires careful calibration of algorithm parameters for each deployment environment.

Future Work: * Develop adaptive preprocessing techniques that adjust based on detected box color and lighting conditions. * Integrate depth information for improved edge detection. * Investigate multi-sensor fusion techniques (e.g., combining vision with LiDAR) to address environmental limitations. * Develop automated parameter optimization systems. * Expand the dataset with more diverse box colors and lighting conditions. * Design standardized deployment guidelines and explore real-time monitoring and adjustment capabilities for seamless integration with warehouse management systems.

Summary updated

Abstract

Automated storage and retrieval systems (AS/RS) play a crucial role in modern logistics, yet effectively monitoring cargo stacking patterns remains challenging. While computer vision and deep learning offer promising solutions, existing methods struggle to balance detection accuracy, computational efficiency, and environmental adaptability. This paper proposes a novel machine vision-based detection algorithm that integrates a pallet surface object detection network (STEGNet) with a box edge detection algorithm. STEGNet's core innovation is the Efficient Gated Pyramid Feature Network (EG-FPN), which integrates a Gated Feature Fusion module and a Lightweight Attention Mechanism to optimize feature extraction and fusion. In addition, we introduce a geometric constraint method for box edge detection and employ a Perspective-n-Point (PnP)-based 2D-to-3D transformation approach for precise pose estimation. Experimental results show that STEGNet achieves 93.49% mAP on our proposed GY Warehouse Box View 4-Dimension (GY-WSBW-4D) dataset and 83.2% mAP on the WSGID-B dataset, surpassing existing benchmarks. The lightweight variant maintains competitive accuracy while reducing the model size by 34% and increasing the inference speed by 68%. In practical applications, the system achieves pose estimation with a Mean Absolute Error within 4 cm and a Rotation Angle Error below 2°, demonstrating robust performance in complex warehouse environments. This research provides a reliable solution for automated cargo stack monitoring in modern logistics systems.
BibTeX
@inproceedings{chen2025vision,
  author = {Chuanjun Chen and Junjie Liu and Haonan Yin and Biqing Huang},
  title = {{A Vision-Based Method for Detecting the Position of Stacked Goods in Automated Storage and Retrieval Systems}},
  booktitle = {Sensors},
  year = {2025},
  doi = {10.3390/s25082623},
}
Figure