Introduction
Modern microservice architectures present a significant management challenge due to their distributed nature and rapid evolution. While Large Language Models (LLMs) have demonstrated potential in automating DevOps tasks, they frequently encounter a "knowledge gap." While an LLM may understand general tools like Kubernetes or Prometheus, it lacks specific knowledge regarding the unique labels, metric names, and undocumented configurations of a particular production environment.
Existing approaches, such as manual prompt engineering or Retrieval-Augmented Generation (RAG), often fail because documentation is frequently outdated or non-existent. ServiceOdyssey addresses this by treating microservice management as a "discovery learning" problem, enabling an autonomic system to explore an unfamiliar environment and teach itself management protocols from scratch.
The ServiceOdyssey Framework
ServiceOdyssey utilizes a modular, multi-agent architecture designed to bridge the gap between general LLM knowledge and service-specific operational requirements. The system is organized into three primary LLM-powered modules that interact across an operational environment and a dedicated data layer.
- Curriculum Builder (CB): This module generates a progression of tasks from "easy to hard" and "observation to action." By prioritizing observation (e.g., checking CPU usage) before intervention (e.g., scaling replicas), the system builds a safe mental model of the environment.
- Execution Planner (EP): The EP translates high-level objectives into executable actions. It employs a hierarchical setup where a High-level Manager coordinates specialized Low-level Agents (e.g., Catalogue or Front-end agents) to resolve sub-tasks.
- Knowledge Curator (KC): Following task execution, the KC extracts successful commands, failure reflections, and system configurations into a permanent Skill Library. This structured memory ensures that service-specific details, such as exact Prometheus label selectors, are retained for future use.
In practice, the framework leverages GPT-4o for routine execution tasks to minimize latency, while utilizing the o1 model for the complex reasoning required in curriculum building and knowledge curation.
Experimental Evaluation
The system was evaluated using the Sock Shop microservice benchmark. The agents underwent five rounds of self-directed exploration (three tasks per round) to build their internal knowledge base. Following this exploration phase, the agents were tested on seven complex operational tasks, such as "Setup Auto Scale" and "Rollback Image."
As shown in the results, there is a clear correlation between autonomous exploration and task success. In the "Initial" state (zero tasks explored), the agent failed significantly on complex tasks like "Setup Auto Scale" (0/3 success). However, by Round 5, the success rate reached 3/3. The learning curve was most pronounced in the early rounds, where agents quickly identified critical system configurations and metric paths that were absent from their general training data.
Key Insights and Discussion
A primary contribution of ServiceOdyssey is the Skill Schema, which categorizes knowledge into executable commands, reflections on logic, and system-specific configurations. This structured approach allows for significantly higher reuse than simple chat histories. Furthermore, the framework proved to be highly cost-effective; onboarding an agent to a new service cost less than $10 in API fees and required under 30 minutes of autonomous operation.
Limitations and Future Work
Despite these successes, several challenges remain. The current prototype was tested on a relatively small-scale microservice; future research must address the scalability of these agents in massive, heterogeneous architectures. Additionally, while the Knowledge Curator includes a validation step to filter hallucinations, the system currently relies on a sandbox environment to prevent potential production outages during the "action" phase of learning. Future iterations may explore parallel exploration strategies to further reduce onboarding time.
Conclusion
ServiceOdyssey demonstrates that self-learning agents can autonomously bridge the knowledge gap in microservice management. By combining curriculum learning with structured knowledge curation, the framework enables LLMs to transition from general-purpose assistants to specialized autonomic managers capable of handling the intricacies of live production environments.
Summary updated