Abstract
Für eine Automatisierungslösung wird ein von künstlicher Intelligenz gestütztes System zur Detektion von unterschiedlichen Objekten sowie deren Positionsmessung entwickelt. Die Objekterkennung erfolgt unter Verwendung eines Jetson Nano von Nvidia. Zur Positionsbestimmung auf einem Förderband wird ein Inkrementaldrehgeber ROD 420 eingesetzt. Der Positionswert wird mit einem selbst entwickelten Messsystem auf einem Arty A7 Field Programmable Gate Array (FPGA) errechnet und zur Sensorfusion an den Jetson Nano übertragen.
Introduction
In modern industrial environments, automated inspection and sorting systems are critical for efficiency. While computer vision systems excel at identifying objects, precisely tracking their physical location on a moving conveyor belt remains a challenge, especially under varying speeds. This research presents a robust sensor fusion approach that combines YOLOv10-based object detection with high-precision hardware encoder data to achieve accurate real-time localization.
System Architecture
The system integrates two primary processing units to handle distinct tasks: the AI-driven vision component and the hardware-level position tracking component.
- Vision Processing (Nvidia Jetson Nano): The system utilizes the YOLOv10 model for object detection. YOLOv10 was selected for its NMS-free (Non-Maximum Suppression) inference, which significantly reduces latency by eliminating a common post-processing bottleneck. This allows the system to maintain high frame rates necessary for industrial lines.
- Position Tracking (Arty A7 FPGA): A ROD 420 incremental encoder measures the conveyor belt's movement. To ensure real-time performance and offload the Jetson Nano, an FPGA decodes the encoder signals. The FPGA calculates the precise position and transmits it to the Jetson Nano via an RS422/UART interface, providing a reliable "physical" coordinate system.
The final sensor fusion occurs on the Jetson Nano, where visual coordinates (y-axis) are merged with encoder-derived longitudinal data (x-axis) to provide a complete spatial profile of each detected object.
Dataset and Training
The model was trained on a specialized dataset of 1,737 images across three categories: screwdrivers (387), watches (1,134), and screws (216). Images were sourced from public repositories and supplemented with custom captures to reflect specific industrial conditions. Data annotation was performed using LabelStudio, and standard augmentations (rotation, mirroring) were applied to enhance model generalization.
Results and Performance
The system achieved high detection accuracy, reaching an F1-Score of 0.94 when utilizing the full dataset. Analysis showed a direct correlation between the volume of training data and the resulting model performance.
Beyond accuracy, the researchers identified a "hardware sweet spot" for training on edge devices. On the Jetson Nano, training time per epoch decreased significantly as GPU memory usage increased up to 3 GB, after which performance gains leveled off. This insight is crucial for optimizing resource allocation in constrained industrial hardware environments.
Conclusion
The proposed sensor fusion system demonstrates that combining high-level AI vision with low-level FPGA-based hardware tracking provides a reliable solution for industrial automation. By offloading position calculations to an FPGA and utilizing low-latency models like YOLOv10, the system achieves the precision required for real-time tracking. Future work will focus on expanding the object classes and exploring direct AI implementation on FPGAs for even greater processing speed.
Summary updated