Dynamic Collaborative Material Distribution System for Intelligent Robots In Smart Manufacturing

Ziren Xiao, Ruxin Xiao, Chang Liu, Xinheng WangarXiv 2025 · DOI · arXiv

Introduction

In modern smart factories, the efficiency of material handling depends on the seamless coordination of multiple autonomous robots. A common scenario involves "loading robots" retrieving materials from warehouses and "distribution robots" stationed at various workshops. To optimize throughput, these agents must converge at a dynamically determined rendezvous point that minimizes total travel distance. This challenge is formally defined as the Dynamic Multiple Sources to Single Destination (DMS-SD) problem.

Conceptual overview of the DMS-SD problem in a grid-based factory layout, showing robots navigating toward a central encounter point.

Traditional approaches often rely on computationally intensive brute-force enumeration, which is ill-suited for the limited processing power of industrial IoT devices like Raspberry Pis. This paper presents a lightweight Deep Reinforcement Learning (DRL) framework that enables real-time, near-optimal coordination for robot swarms.

Proposed Framework

The researchers developed a DRL-based approach utilizing Proximal Policy Optimization (PPO). The system architecture is designed for scalability and efficiency in industrial environments, utilizing a Centralized Training, Decentralized Execution (CTDE) paradigm.

System architecture showing the interaction between the environment, central server for training, and the DRL agent structure.

Key Methodological Innovations

  • Centralized Training, Decentralized Execution (CTDE): Agents are trained in a global simulation environment where they have access to full state information. Once deployed, each robot operates independently using a local copy of the trained model, making decisions based on its specific observations.
  • Homogeneous Model Sharing: To accelerate convergence and ensure unified swarm logic, all robots share the same neural network weights. This "shared brain" approach allows the system to scale to different swarm sizes without retraining.
  • Potential-Based Reward Function: The reward mechanism guides robots toward the geometric centroid of the swarm. By framing the coordination problem as a target-seeking task toward a moving center, the complexity of multi-agent interaction is significantly reduced.
  • Action Masking: To streamline the learning process, the system incorporates a hard-coded masking mechanism. It identifies and blocks invalid moves—such as collisions with walls or other robots—allowing the RL agent to focus exclusively on high-level strategic coordination rather than basic obstacle avoidance.

Experimental Evaluation

The framework was evaluated using NVIDIA Isaac Sim in grid-based factory layouts ranging from 50x50 to 70x70 units. The maps included a 5% density of static obstacles and a 2% density of dynamic obstacles, such as moving humans. Tests were conducted with swarms ranging from 2 to 12 robots.

Key Results

  • Computational Efficiency: The DRL model generates movement decisions in milliseconds (~30ms on a desktop, <100ms on a Raspberry Pi 4), representing a 100x speedup over traditional non-learning solvers.
  • Path Optimality: The paths generated by the DRL agents were found to be nearly identical to the mathematical optimum, typically differing by fewer than five steps across an entire trajectory.
  • Robustness and Flexibility: The system effectively handles complex scenarios, such as escaping local optima caused by U-shaped obstacles. Furthermore, a model trained on a 12-robot swarm can successfully control smaller groups (e.g., 4 robots) using zero-padding in the state space, demonstrating high transferability.

Conclusion

This research demonstrates that a lightweight DRL framework with action masking and shared weights can solve complex multi-agent coordination problems in real-time. While currently validated on grid-based maps, future work will focus on extending these capabilities to continuous space navigation and incorporating additional constraints such as varying robot speeds and battery management.

Summary updated

Abstract

The collaboration and interaction of multiple robots have become integral aspects of smart manufacturing. Effective planning and management play a crucial role in achieving energy savings and minimising overall costs. This paper addresses the real-time Dynamic Multiple Sources to Single Destination (DMS-SD) navigation problem, particularly with a material distribution case for multiple intelligent robots in smart manufacturing. Enumerated solutions, such as in \cite{xiao2022efficient}, tackle the problem by generating as many optimal or near-optimal solutions as possible but do not learn patterns from the previous experience, whereas the method in \cite{xiao2023collaborative} only uses limited information from the earlier trajectories. Consequently, these methods may take a considerable amount of time to compute results on large maps, rendering real-time operations impractical. To overcome this challenge, we propose a lightweight Deep Reinforcement Learning (DRL) method to address the DMS-SD problem. The proposed DRL method can be efficiently trained and rapidly converges to the optimal solution using the designed target-guided reward function. A well-trained DRL model significantly reduces the computation time for the next movement to a millisecond level, which improves the time up to 100 times in our experiments compared to the enumerated solutions. Moreover, the trained DRL model can be easily deployed on lightweight devices in smart manufacturing, such as Internet of Things devices and mobile phones, which only require limited computational resources.
BibTeX
@misc{xiao2025dynamic,
  author = {Ziren Xiao and Ruxin Xiao and Chang Liu and Xinheng Wang},
  title = {{Dynamic Collaborative Material Distribution System for Intelligent Robots In Smart Manufacturing}},
  year = {2025},
  eprint = {2506.11723},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.11723},
}
Figure