Abstract
In Digital Twin Mobile Edge Networks (DTMEN), edge servers must balance the computational demands of processing offloaded vehicular tasks with the overhead of maintaining real-time virtual replicas (Digital Twins) of those vehicles. This paper introduces TMTPRCS, a Multi-Agent Reinforcement Learning (MADRL) algorithm designed to collaboratively schedule resources for both maintenance and processing. By employing a Centralized Training, Distributed Execution (CTDE) framework, TMTPRCS significantly improves resource utility and deadline adherence, outperforming standard reinforcement learning baselines by up to 83%.
The Challenge: Balancing Twins and Tasks
As autonomous and connected vehicles become more prevalent, they increasingly rely on Mobile Edge Computing (MEC) to handle heavy tasks like high-definition video processing and navigation. However, for an edge server to make intelligent decisions, it must maintain a Digital Twin (DT)—a synchronized virtual model—of each vehicle.
This creates a critical resource competition. Both the synchronization of the twin (updating velocity, location, and engine data) and the execution of offloaded tasks consume the server's finite CPU power. Most existing research treats these as isolated problems, but in a real-world DTMEN, they occur simultaneously. The core problem addressed here is how to divide these limited resources so that digital twins remain accurate without causing excessive delays in the vehicles' primary computational tasks.
The TMTPRCS Algorithm
The authors frame this resource allocation as a Multi-Agent Markov Decision Process (MDP), where each vehicle acts as an intelligent agent. To solve this, they propose the Twin Maintenance and Task Processing Resource Collaborative Scheduling (TMTPRCS) algorithm.
TMTPRCS is built on an Actor-Critic framework with two primary innovations:
- Centralized Training, Distributed Execution (CTDE): During the training phase, the server uses global information to optimize the policy. Once deployed, individual vehicles can execute their resource requests locally in real-time based on their specific state.
- Adaptive $\epsilon$-greedy Exploration: Unlike standard algorithms that use fixed noise for exploration, TMTPRCS dynamically shifts from exploration to exploitation as training progresses. This ensures the model converges on effective policies without losing them to random noise late in the training cycle.
Experimental Setup and Results
The study utilized a synthetic simulation environment modeled after a 3-lane highway with realistic vehicular networking parameters (20MHz bandwidth, V2I links). Tasks ranged from 1000 to 1500 bytes, while DT maintenance updates were smaller (10-200 bytes), with strict delay deadlines of 500ms and 100ms, respectively.
Key Findings
- Superior Performance: TMTPRCS achieved reward values 6.42% higher than SAC, 11.15% higher than PPO, and 83.42% higher than MATD3.
- Resource Efficiency: The algorithm maintained near 100% resource utilization in optimal scenarios, whereas baseline algorithms often left resources idle or failed to meet deadlines.
- The Capacity Limit: Results indicated that resource utility peaks when a server manages approximately 6 vehicles. Beyond this point, the competition for CPU cycles becomes so intense that overall system utility drops, highlighting a physical limit to edge node management.
Conclusion and Future Directions
TMTPRCS demonstrates that Digital Twin maintenance is not "free" and must be accounted for in resource allocation strategies. While highly effective in a single-server highway model, future work is needed to address twin migration—the process of moving a virtual model between edge servers as a vehicle travels—and more complex mobility dynamics like sudden braking or lane changes.
Summary updated