Brevity is the Soul of Wit: Condensing Code Changes to Improve Commit Message Generation

Hongyu Kuang, Ning Zhang, Hui Gao, Xin Zhou, Wesley K. G. Assunção, Xiaoxing Ma, Dong Shao, Guoping Rong, He Zhang

Introduction

Writing clear, descriptive commit messages is a fundamental yet often neglected aspect of software maintenance. While automated Commit Message Generation (CMG) tools aim to assist developers, they frequently struggle with the "semantic gap"—the distance between raw code modifications and the high-level intent behind them. Raw git diffs are inherently noisy, containing implementation details that can overwhelm Large Language Models (LLMs) and lead to poor documentation quality.

The CONTENT Framework

To address this, researchers have introduced CONTENT (Condensing cOde chaNges for commiT mEssage geNeraTion). The core philosophy of CONTENT is that providing an LLM with distilled, high-signal data is more effective than feeding it raw, voluminous code diffs. The framework utilizes a three-part condensation pipeline to transform messy diffs into structured templates.

The generic CONTENT template structure used to organize summarized changes, comments, and identifiers.

The Condensation Pipeline

  1. Summarized Code Changes: The framework adapts ChangeScribe to convert raw diffs into concise natural language summaries. This replaces lines of boilerplate code with high-level descriptions like "Add a constructor method."
  2. Elicited Comments: By using regular expressions to extract JavaDoc, TODOs, and inline comments, CONTENT captures the developer's explicit rationale, which is often missing from the code itself.
  3. Emphasized Code Identifiers: Key identifiers (such as class and method names) are extracted and processed using CamelCase splitting. This helps the model understand the domain-specific context by treating code identifiers as natural language words.
A concrete example of the CONTENT framework distilling a code change into summarized parts, elicited comments, and emphasized identifiers.

Experimental Results

The researchers fine-tuned CodeLlama-7B using Low-Rank Adaptation (LoRA) on the refined CoDiSum dataset, which consists of over 80,000 high-quality Java code-message pairs.

CONTENT demonstrated substantial performance gains over six state-of-the-art baselines, including NNGen and FIRA. Evaluation metrics showed improvements of 51.7% in BLEU-Norm, 78.7% in METEOR, and 62.5% in ROUGE-L. Furthermore, a blind study involving industrial Java developers rated CONTENT’s outputs as the most semantically accurate and useful, confirming that the condensed representation successfully bridges the semantic gap.

Conclusion

The success of CONTENT proves that when it comes to automated documentation, brevity is indeed the soul of wit. By pre-digesting complex code changes into human-centric summaries, the framework allows LLMs to focus on the rationale and intent of a change, resulting in superior commit messages that better serve the needs of software development teams.

Summary updated

BibTeX
@misc{kuang2025brevity,
  author = {Hongyu Kuang and Ning Zhang and Hui Gao and Xin Zhou and Wesley K. G. Assunção and Xiaoxing Ma and Dong Shao and Guoping Rong and He Zhang},
  title = {{Brevity is the Soul of Wit: Condensing Code Changes to Improve Commit Message Generation}},
  year = {2025},
  eprint = {2509.15567},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.15567},
}
Figure