Hallucination to Consensus: Multi-Agent LLMs for End-to-End Test Generation with Accurate Oracles

Qinghua Xu, Guancheng Wang, Lionel C. Briand, Kui LiuarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces CANDOR, a novel multi-agent LLM framework that automates the generation of complete JUnit tests, including both code and accurate assertions (oracles). By employing a "panel discussion" among specialized LLM agents, CANDOR effectively mitigates LLM hallucinations and produces more reliable test oracles than existing methods, even for faulty code.

What Problem Does This Paper Address?

Writing comprehensive unit tests for software is crucial for ensuring code quality, but it's a time-consuming and labor-intensive process for developers. While automated test generation tools exist, they often struggle with generating human-readable test prefixes and, more critically, accurate test oracles. Existing LLM-based approaches for oracle generation often require fine-tuning on specific datasets, which can be costly and inflexible, or they rely on regression oracles that only detect behavioral changes, not functional correctness. This paper tackles the challenge of generating end-to-end JUnit tests with accurate, specification-based oracles directly from natural language descriptions, even when the source code itself contains bugs.

Methods

CANDOR employs a multi-agent approach, breaking down the complex task of test generation into specialized roles. The framework consists of three main steps:

  • Initialization: An "Initializer" LLM agent generates an initial, syntactically correct test file. This step is optional if a starting test file is provided.
  • Test Prefix Generation: A team of agents (Planner, Tester, Inspector) iteratively generates and refines test cases to improve code coverage. The "Planner" identifies coverage gaps, the "Tester" writes the test code, and the "Inspector" checks for execution errors. This step produces test prefixes with tentative oracles derived from the source code.
  • Oracle Fixing: This is the core innovation. To address hallucinations and ensure oracle accuracy, CANDOR uses a "panel discussion" strategy. Multiple "Panelist" LLM agents, powered by reasoning LLMs, independently evaluate tentative oracles against natural language descriptions. A "dual-LLM pipeline" (Panelist + Interpreter) extracts concise insights from the verbose reasoning of each Panelist. Finally, a "Curator" agent aggregates these interpretations to reach a consensus and generate the final, accurate oracle. This consensus mechanism is key to mitigating LLM hallucinations.

Datasets

The authors evaluated CANDOR on two datasets:

  • HumanEvalJava: A well-established benchmark for Java unit test generation.
  • LeetCodeJava: A new dataset constructed from programming problems on the LeetCode platform, specifically designed to include programs with varying difficulty levels and more complex structural characteristics. This dataset aims to test CANDOR's robustness.

Key Insights and Ideas

  • Consensus over Majority: The "panel discussion" approach, where LLMs explain their reasoning and reach a consensus, is more robust than simple majority voting for generating accurate oracles. This is because it allows for cross-validation of reasoning logic, inputs, and outputs, effectively filtering out hallucinations.
  • Specialized Agents for Complex Tasks: Decomposing the complex task of test generation into specialized LLM agents (Planner, Tester, Inspector, etc.) prevents interference and improves overall effectiveness, unlike single-LLM approaches that can become confused.
  • Natural Language Descriptions are Powerful: CANDOR's success in generating accurate oracles, even from faulty code, highlights the power of using natural language descriptions as a source of truth, rather than relying solely on potentially buggy source code.
  • Mitigating LLM Verbosity: The dual-LLM pipeline (Panelist + Interpreter) effectively addresses the "overthinking phenomenon" in reasoning LLMs, extracting concise and structured outputs without sacrificing the depth of reasoning.

Key Results

  • Test Prefix Quality: CANDOR achieved comparable line and branch coverage to EvoSuite, a leading test prefix generator, while significantly outperforming it in mutation score (indicating better bug-detection capability). CANDOR also substantially outperformed LLM-Empirical in test prefix quality.
  • Oracle Correctness: CANDOR achieved significantly higher oracle correctness than LLM-Empirical and TOGLL (a state-of-the-art fine-tuned oracle generator) across both correctly implemented and faulty code scenarios. For example, CANDOR outperformed TOGLL by at least 21.1 percentage points in oracle correctness.
  • Robustness: CANDOR demonstrated stable performance across datasets with increasing complexity, suggesting its robustness.
  • Ablation Studies: Removing the "Planner" agent significantly reduced test prefix quality, while removing the "Requirement Engineer" or the "panel discussion" primarily impacted oracle correctness, underscoring the critical roles of these components.

Limitations and Future Work

  • LLM Choice: The evaluation used specific LLMs (Llama 3.1 70B and DeepSeek R1). While these were chosen for their capabilities, other LLMs might yield different performance.
  • Data Leakage: There's a potential threat of data leakage if the evaluation datasets were included in the LLMs' pretraining data. The authors mitigated this by using mutation testing, which generates synthetic bugs unlikely to be in pretraining data.
  • Dependencies: The current framework focuses on Java methods without user-defined classes or external libraries. Future work aims to extend CANDOR to handle complex programs with dependencies by introducing dedicated agents for retrieval and mocking.
  • Project-Level Testing: The authors plan to extend CANDOR to project-level test generation.
  • Advanced Panel Discussion: Exploring more sophisticated panel discussion strategies is also a future direction.

Summary updated

BibTeX
@misc{xu2025hallucination,
  author = {Qinghua Xu and Guancheng Wang and Lionel C. Briand and Kui Liu},
  title = {{Hallucination to Consensus: Multi-Agent LLMs for End-to-End Test Generation with Accurate Oracles}},
  year = {2025},
  eprint = {2506.02943},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.02943},
}
Figure