SGCR: A Specification-Grounded Framework for Trustworthy LLM Code Review

Kai Wang, Bingcheng Mao, Shuai Jia, Yujie Ding, Dongming Han, Tianyi Ma, Bin CaoarXiv 2025 · DOI · arXiv

Abstract

Large Language Models (LLMs) show significant promise in code understanding but often fail in enterprise environments due to a lack of domain context, stochasticity, and poor controllability. This paper introduces SGCR (Specification-Grounded Code Review), a framework designed to ground LLM reasoning in human-authored specifications. By utilizing a dual-pathway architecture—Explicit Specification Injection and Implicit Specification Discovery—SGCR bridges the gap between the creative reasoning of LLMs and the rigorous requirements of industrial software development. In a live industrial deployment, SGCR achieved a 42% developer adoption rate, nearly doubling the performance of vanilla LLM baselines.

Introduction

While LLMs are increasingly used for code generation, their application in code review is hampered by several "trust killers." These include a lack of proprietary domain context, inconsistent outputs (stochasticity), and a tendency to focus on stylistic "nitpicks" rather than critical logic or security flaws. Conversely, traditional static analysis tools are reliable but often miss complex logical errors found by human reviewers.

SGCR addresses these challenges by transforming the LLM from a "stochastic parrot" into a grounded reviewer. The framework provides the model with a "rulebook" or specification, ensuring that suggestions are relevant, explainable, and aligned with project-specific standards.

Methodology: The Dual-Pathway Architecture

The core innovation of SGCR is its dual-pathway approach, which balances deterministic rule-checking with heuristic issue discovery to avoid the "checklist trap"—where a model ignores valid bugs simply because they aren't explicitly listed.

Figure 1: The SGCR Dual-Pathway Architecture, featuring Explicit Specification Injection and Implicit Specification Discovery modules.

1. Explicit Specification Injection

This "top-down" pathway utilizes a library of human-authored rules. To manage long specifications without diluting the model's attention, SGCR segments rules into smaller chunks. An ensemble of LLM reviewers evaluates the code against these segments, and a second-order LLM Aggregator synthesizes the findings into a consistent, de-duplicated report.

2. Implicit Specification Discovery

This "bottom-up" pathway allows for more creative analysis. A Proposer LLM first identifies potential issues through free-form review. These "hypothetical issues" are then converted into vector embeddings to retrieve relevant rules from a database. Finally, a Verifier LLM confirms the validity of the issue based on the retrieved evidence, ensuring the suggestion remains grounded in actual project requirements.

Experimental Setup and Results

The researchers evaluated SGCR using a library of 140 Java-specific rules covering security, performance, and business logic. The system was deployed in a live production environment at HiThink Research.

The primary metric for success was the Adoption Rate—the percentage of LLM-generated suggestions that developers actually implemented in the codebase.

Figure 3: Ablation study showing the impact of different SGCR modules on the developer adoption rate.

As shown in the results, the full SGCR framework achieved a 42% adoption rate, a significant improvement over the 22% achieved by a vanilla LLM baseline. Ablation studies revealed that while the Explicit pathway provided the most substantial performance boost (increasing adoption to 37%), the Implicit pathway served as a vital safety net, catching an additional 5% of issues that fell outside the primary rule segments.

Key Insights and Discussion

  • Stability through Ensembles: Using multiple model instances and a dedicated Aggregator significantly reduced hallucinations and the inconsistency typical of single-shot prompts.
  • Educational Value: Qualitative feedback indicated that junior developers used SGCR as a learning tool, as every suggestion was linked back to an authoritative project standard, providing clear rationale for the requested changes.
  • The Propose-then-Verify Strategy: By allowing the LLM to "think freely" before grounding it in the Implicit path, the framework successfully avoided the analytical suppression often caused by providing strict rules upfront.

Conclusion and Future Work

SGCR demonstrates that grounding LLMs in explicit specifications is essential for enterprise-grade code review. While the framework significantly improves trust and adoption, challenges remain regarding the manual effort required to maintain rule libraries. Future work will focus on creating more interactive interfaces for developer-LLM clarification and automating the update process for specification libraries as business logic evolves.

Summary updated

BibTeX
@misc{wang2025sgcr,
  author = {Kai Wang and Bingcheng Mao and Shuai Jia and Yujie Ding and Dongming Han and Tianyi Ma and Bin Cao},
  title = {{SGCR: A Specification-Grounded Framework for Trustworthy LLM Code Review}},
  year = {2025},
  eprint = {2512.17540},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.17540},
}
Figure