TL;DR
EREBOTS 4.0 is an open-source, hybrid-AI framework designed to standardize how LLM-based agents communicate and use tools. By leveraging the Agent2Agent (A2A) and Model Context Protocol (MCP) specifications, it allows researchers to build modular, reproducible conversational assistants without having to rebuild the underlying infrastructure for every new study.
What Problem Does This Paper Address?
The field of multi-agent systems (MAS) has shifted from rule-based symbolic agents to sub-symbolic agents powered by Large Language Models (LLMs). However, this shift has introduced a "plumbing" problem: researchers often spend more time building the substrate—discovery, transport, identity management, and conversation state—than they do on the actual agent logic.
Without a common framework, results are difficult to replicate, and it is nearly impossible to swap out a single component (like a routing strategy) to measure its impact in isolation. EREBOTS 4.0 aims to provide this missing "neuro-symbolic" frontier, pairing the reasoning of LLMs with the reliability of deterministic tools through standardized protocols.
Methods
The framework employs a star-topology architecture where a central Orchestrator manages the interaction between the user and various Specialist Agents.
- Hybrid-AI Approach: LLMs are used for high-level planning and routing, while deterministic tools handle exact computations (e.g., unit conversions or database updates).
- Protocol Integration:
- A2A (Agent2Agent): Used for inter-agent communication. Adding a new agent—whether local or remote—is as simple as adding its URL to a configuration file.
- MCP (Model Context Protocol): Standardizes how agents discover and invoke tools, ensuring that tool-using specialists can operate across different hosts.
- Identity & Persistence: The framework integrates OpenID Connect (OIDC) via Keycloak for user authentication. It propagates user identity and consent flags to all specialists, ensuring data privacy through server-side pinning of user IDs.
- Execution Tracing: It records every routing decision, tool call, and failure, providing a detailed "trace" that is essential for debugging the often-opaque reasoning of LLM-based systems.
Datasets
The paper does not introduce a traditional training dataset. Instead, it focuses on conformance testing and demonstrator scenarios.
- A2A Technology Compatibility Kit (TCK): The primary benchmark used to evaluate the framework's adherence to the A2A protocol.
- Demonstrator Agents: The authors provide two specialists:
PersonalAgent(for managing user profile data) andConversionAgent(for exact unit conversions).
Key Insights and Ideas
- Protocol-First Modularity: By adhering to A2A and MCP, EREBOTS 4.0 makes agents truly "plug-and-play." Two different research groups can compose each other's agents without sharing a codebase.
- Resumable "Parked" Tasks: The framework handles "clarification loops" (where an agent needs more info from the user) by parking the task state. These tasks can survive server restarts and are securely tied to the original user's identity.
- Deterministic Reliability: By offloading math and data retrieval to MCP tools rather than relying on LLM generation, the system avoids common "hallucinations" in factual or numerical tasks.
- Infrastructure as a Service: The framework takes over the "boring" parts of app development (login, context storage, shared transcripts across frontends), allowing researchers to focus strictly on agent behavior.
Key Results
The framework's performance was validated through an automated suite of 567 tests and the official A2A TCK.
- Protocol Conformance: EREBOTS 4.0 passed 97 requirements of the official A2A TCK with zero failures (see Figure 3).
- SDK Improvements: The rigorous testing process identified and corrected two "MUST" requirement deviations in the official
a2a-sdk(related to error code handling and terminal task subscriptions). - Tech Stack: The implementation is built on modern standards including Python 3.13, FastAPI, and SQLAlchemy, supporting both PostgreSQL and SQLite (see Figure 2).
Limitations and Future Work
The authors acknowledge several areas for improvement:
- Concurrency: The framework currently only supports one queued waiting task at a time; a second clarification request in a fan-out scenario must wait for the first to expire.
- Consent Enforcement: While consent flags are stored and served, the framework does not yet automatically block operations if consent is missing; this logic currently resides within the specialists.
- Agent Negotiation: Currently, the orchestrator makes all routing decisions. Future work will explore "contract net" protocols where agents bid on tasks based on their self-reported suitability.
- Model Dependency: While the framework's mechanics are verified with stubbed replies, the ultimate success of the assistant still depends on the quality of the underlying LLM used for routing and planning.
Summary updated