Abstract
The Text-to-SQL task aims to automatically yield SQL queries according to user text questions. To address this problem, we propose a Cooperative SQL Generation framework based on Multi-functional Agents (CSMA) through information interaction among large language model (LLM) based agents who own part of the database schema separately. Inspired by the collaboration in human teamwork, CSMA consists of three stages: 1) Question-related schema collection, 2) Question-corresponding SQL query generation, and 3) SQL query correctness check. In the first stage, agents analyze their respective schema and communicate with each other to collect the schema information relevant to the question. In the second stage, agents try to generate the corresponding SQL query for the question using the collected information. In the third stage, agents check if the SQL query is created correctly according to their known information. This interaction-based method makes the question-relevant part of database schema from each agent to be used for SQL generation and check. Experiments on the Spider and Bird benchmark demonstrate that CSMA achieves a high performance level comparable to the state-of-the-arts, meanwhile holding the private data in these individual agents.
Introduction: Addressing the "God-Eye View" Problem
Most existing Text-to-SQL systems assume a "god-eye view," where a single model has access to the entire database schema simultaneously. However, in real-world enterprise environments, databases are often massive, complex, and segmented across different departments or security tiers for privacy and scalability reasons.
Feeding an entire massive schema into an LLM can overwhelm its context window with noise and violates security protocols where no single user is permitted to see the full data map. The CSMA framework addresses this by modeling the task as a decentralized decision-making process among siloed agents.
The CSMA Framework
CSMA utilizes multiple LLM agents, each owning only a fragment of the database, to collaborate through a shared "Global State." This architecture allows agents to contribute their specific knowledge without exposing their entire private schema.
The Three-Stage Process
- Question-related Schema Collection: Agents maintain a "Global Schema" that acts as a shared bulletin board. Each agent merges their private schema with the current global state, extracts relevant parts, and updates the board. A retention mechanism ensures vital context is preserved even if an agent initially deems a column irrelevant.
- Question-corresponding SQL Generation: A designated "working agent" uses the enriched global schema to write the SQL query. Different agents provide unique perspectives based on their specific data segments.
- SQL Correctness Check: This "peer review" phase passes the generated SQL to the next agent in rotation. This agent verifies the query against their own schema for errors. If it fails, the cycle repeats with updated information; otherwise, the process terminates.
Experimental Evaluation
The framework was evaluated on the Spider and BIRD benchmarks, representing diverse domains and large-scale database challenges.
- Competitive Performance: On the Spider Dev set, the "TwoPart" CSMA approach (two agents with partial schemas) achieved an execution accuracy of 75.92%, slightly outperforming the centralized "OneAll" baseline (75.15%).
- Robustness on BIRD: On the more difficult BIRD benchmark, CSMA reached 49.28% accuracy, again surpassing the single-agent centralized approach (48.89%).
- Ablation Results: Removing communication or peer-review components led to significant performance drops, confirming that cooperation is essential for accuracy in segmented environments.
Key Insights and Conclusion
A primary finding of this research is that segmenting the database can actually improve performance by reducing noise; the "TwoPart" setting often outperformed the "OneAll" setting. Furthermore, the iterative peer-review process effectively catches hallucinations and errors that a single agent might overlook. While challenges remain regarding communication overhead as the number of agents increases, CSMA demonstrates that decentralized, cooperative agents can match the utility of centralized systems while respecting data segmentation.
Summary updated