Collaborative Evolution of Intelligent Agents in Large-Scale Microservice Systems

Yilin Li, Song Han, Sibo Wang, Ming Wang, Renzi Meng

Abstract

This paper proposes an intelligent service optimization method based on a multi-agent collaborative evolution mechanism to address governance challenges in large-scale microservice architectures. These challenges include complex service dependencies, dynamic topology structures, and fluctuating workloads. The method models each service as an agent and introduces graph representation learning to construct a service dependency graph. This enables agents to perceive and embed structural changes within the system. Each agent learns its policy based on a Markov Decision Process. A centralized training and decentralized execution framework is used to integrate local autonomy with global coordination. To enhance overall system performance and adaptability, a game-driven policy optimization mechanism is designed. Through a selection-mutation process, agent strategy distributions are dynamically adjusted. This supports adaptive collaboration and behavioral evolution among services. Under this mechanism, the system can quickly respond and achieve stable policy convergence when facing scenarios such as sudden workload spikes, topology reconfigurations, or resource conflicts. To evaluate the effectiveness of the proposed method, experiments are conducted on a representative microservice simulation platform. Comparative analyses are performed against several advanced approaches, focusing on coordination efficiency, adaptability, and policy convergence performance. Experimental results show that the proposed method outperforms others in several key metrics. It significantly improves governance efficiency and operational stability in large-scale microservice systems. The method demonstrates strong practical value and engineering feasibility.

Introduction

As software architectures transition from monolithic blocks to hundreds of independent microservices, the complexity of managing these systems grows exponentially. Traditional centralized schedulers often struggle to keep up with "microservice sprawl"—a phenomenon where service dependencies change frequently, workloads fluctuate wildly, and the sheer number of components makes global optimization computationally impossible. This paper addresses the critical challenge of intelligent service governance, specifically focusing on maintaining system stability and resource efficiency during sudden traffic bursts or structural reconfigurations without relying on a slow, single point of control.

Methodology

The researchers propose a multi-agent collaborative evolution mechanism that treats every microservice as an autonomous agent. The framework rests on three technical pillars:

  1. Graph Representation Learning: Using Graph Convolutional Networks (GCNs), the system builds a dynamic "map" of service dependencies. This allows agents to perceive how changes in one service (like a latency spike) might ripple through the rest of the network. By embedding the system topology directly into the agents' decision-making process, the agents react to the context of their neighbors rather than just local metrics.
  2. Centralized Training, Decentralized Execution (CTDE): Agents are trained together to understand global goals but execute their policies locally. This provides the best of both worlds: global coordination during the learning phase and lightning-fast local response times during operation.
  3. Evolutionary Game-Driven Optimization: Instead of standard reinforcement learning, the authors apply evolutionary principles including selection and mutation. Strategies that lead to better system health are "selected" and propagated, while inefficient ones are phased out. This helps the system converge on stable policies much faster than traditional methods.
The multi-agent collaborative evolution framework architecture.

Experimental Evaluation

The study utilizes the MICROBENCH dataset, a specialized benchmark for microservice performance evaluation. It contains real-world operational traces, deployment topologies, CPU/memory usage logs, and interface invocation frequencies. The experiments simulated various scales (from dozens to hundreds of services) and failure modes common in e-commerce and social networking applications.

The "Goldilocks Zone" of Agent Density

One of the most significant findings was the relationship between the number of agents and global optimality. The system does not scale linearly in terms of performance; rather, it exhibits a peak efficiency range.

Impact of agent quantity on global policy optimality, showing a peak between 60 and 80 agents.

As illustrated in the figure, the system performed best with 60–80 agents. Beyond this range, "coordination bottlenecks" emerged, where too many autonomous actors began to interfere with one another, leading to a slight decline in global optimality.

Key Results and Performance

The proposed method was compared against several state-of-the-art baselines, including DDPG-MARL and E-MADDPG. The results highlight the framework's superior efficiency:

  • Coordination Efficiency: The proposed model achieved a score of 0.912, significantly higher than the best baseline (0.844).
  • Convergence Speed: The system reached a stable policy in 48.3 seconds, nearly 30% faster than standard multi-agent reinforcement learning approaches.
  • Resilience to Bursts: In simulated "burst traffic" tests, the system kept latency below 0.7 seconds and demonstrated a rapid recovery post-peak, proving it could effectively reallocate resources on the fly.

Conclusion and Future Work

While the results are promising, the authors acknowledge that real-world production environments often have "noisier" data and more unpredictable human-in-the-loop elements than the MICROBENCH simulation. Furthermore, the computational overhead of using GCNs for dynamic graph embeddings could become a bottleneck at a massive scale (thousands of services).

Future research will explore "approximate embeddings" to reduce computational costs and investigate "cross-system policy transfer"—the ability for agents to apply learned strategies from one microservice environment to a completely different one.

Summary updated

Abstract

This paper proposes an intelligent service optimization method based on a multi-agent collaborative evolution mechanism to address governance challenges in large-scale microservice architectures. These challenges include complex service dependencies, dynamic topology structures, and fluctuating workloads. The method models each service as an agent and introduces graph representation learning to construct a service dependency graph. This enables agents to perceive and embed structural changes within the system. Each agent learns its policy based on a Markov Decision Process. A centralized training and decentralized execution framework is used to integrate local autonomy with global coordination. To enhance overall system performance and adaptability, a game-driven policy optimization mechanism is designed. Through a selection-mutation process, agent strategy distributions are dynamically adjusted. This supports adaptive collaboration and behavioral evolution among services. Under this mechanism, the system can quickly respond and achieve stable policy convergence when facing scenarios such as sudden workload spikes, topology reconfigurations, or resource conflicts. To evaluate the effectiveness of the proposed method, experiments are conducted on a representative microservice simulation platform. Comparative analyses are performed against several advanced approaches, focusing on coordination efficiency, adaptability, and policy convergence performance. Experimental results show that the proposed method outperforms others in several key metrics. It significantly improves governance efficiency and operational stability in large-scale microservice systems. The method demonstrates strong practical value and engineering feasibility.
BibTeX
@misc{li2025collaborative,
  author = {Yilin Li and Song Han and Sibo Wang and Ming Wang and Renzi Meng},
  title = {{Collaborative Evolution of Intelligent Agents in Large-Scale Microservice Systems}},
  year = {2025},
  eprint = {2508.20508},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.20508},
  howpublished = {arXiv},
}
Figure