Code2Que: A Tool for Improving Question Titles from Mined Code Snippets in Stack Overflow

Zhipeng Gao, Xin Xia, David Lo, John C. Grundy, Yuan-Fang Li

Abstract

Stack Overflow is one of the most popular technical Q&A sites used by software developers. Seeking help from Stack Overflow has become an essential part of software developers' daily work for solving programming-related questions. Although the Stack Overflow community has provided quality assurance guidelines to help users write better questions, we observed that a significant number of questions submitted to Stack Overflow are of low quality. In this paper, we introduce a new web-based tool, Code2Que, which can help developers in writing higher quality questions for a given code snippet. Code2Que consists of two main stages: offline learning and online recommendation. In the offline learning phase, we first collect a set of good quality <code snippet, question> pairs as training samples. We then train our model on these training samples via a deep sequence-to-sequence approach, enhanced with an attention mechanism, a copy mechanism and a coverage mechanism. In the online recommendation phase, for a given code snippet, we use the offline trained model to generate question titles to assist less experienced developers in writing questions more effectively. At the same time, we embed the given code snippet into a vector and retrieve the related questions with similar problematic code snippets.

Introduction

Stack Overflow (SO) is a critical resource for modern software development, yet its utility is often hampered by the quality of user-submitted questions. Vague or poorly articulated titles frequently fail to attract the necessary expertise, leaving developers without solutions. This "cold start" problem is particularly acute for less experienced users who may lack the technical vocabulary to describe a bug or are non-native English speakers. While existing research has focused on identifying low-quality posts, Code2Que proposes a proactive solution: a tool that automatically generates descriptive, high-quality titles directly from code snippets.

Methodology

Code2Que operates through a two-stage framework: offline learning and online recommendation.

Offline Learning

The system was trained on a dataset derived from the September 2019 Stack Overflow data dump, encompassing over one million high-quality <code snippet, question> pairs across five languages: Python, Java, JavaScript, C#, and SQL. The core architecture utilizes a deep sequence-to-sequence (seq2seq) model. To address the specific complexities of source code, three enhancements were integrated:

  • Attention Mechanism: Directs the model to focus on the most salient parts of a code snippet.
  • Copy Mechanism: Crucial for technical accuracy, this allows the model to extract rare tokens (e.g., specific function names) directly from the input code.
  • Coverage Mechanism: Mitigates linguistic repetition, ensuring the generated titles are concise and natural.

Online Recommendation

In the recommendation phase, users provide a code snippet to the tool. The trained model generates a new title, while the system simultaneously embeds the code into a vector space to retrieve the five most similar existing questions from Stack Overflow, providing immediate context and potential answers.

Overview of the Code2Que architecture showing the transition from offline training to online recommendation.

Results and Discussion

The effectiveness of Code2Que was evaluated through a user study involving 50 "low-quality" Python and Java question pairs. Human evaluators judged the generated titles against the original human-written ones based on three metrics: Clearness (readability), Fitness (relevance to code), and Willingness to Respond (likelihood of attracting expert help).

The results demonstrated a significant improvement over original low-quality titles. In Python, Code2Que titles outperformed human titles in Clearness (52.4%), Fitness (55.2%), and Willingness to Respond (61.2%). Similar trends were observed in Java, where the tool consistently outperformed or tied with human-written titles.

A key insight from the study is the importance of the Copy Mechanism. In technical contexts, the specific name of a library or function is often the most critical component of a title. Standard NLP models often struggle with these "out-of-vocabulary" terms. As shown in the comparative examples, the inclusion of the Copy Mechanism allows the model to correctly identify and include specific methods like get_client_ip, which significantly boosts the professional quality of the generated question.

Example showing how different model enhancements (Attention, Coverage, Copy) affect the quality of the generated question.

Conclusion

Code2Que represents a significant step toward automating the improvement of technical Q&A quality. By treating code as a structured language and applying specialized seq2seq enhancements, the tool helps bridge the gap between a developer's technical struggle and the community's ability to provide assistance. Future work will explore incorporating additional context, such as post body text and user-provided tags, to further refine the accuracy and relevance of generated questions.

Summary updated

Abstract

Stack Overflow is one of the most popular technical Q&A sites used by software developers. Seeking help from Stack Overflow has become an essential part of software developers' daily work for solving programming-related questions. Although the Stack Overflow community has provided quality assurance guidelines to help users write better questions, we observed that a significant number of questions submitted to Stack Overflow are of low quality. In this paper, we introduce a new web-based tool, Code2Que, which can help developers in writing higher quality questions for a given code snippet. Code2Que consists of two main stages: offline learning and online recommendation. In the offline learning phase, we first collect a set of good quality <code snippet, question> pairs as training samples. We then train our model on these training samples via a deep sequence-to-sequence approach, enhanced with an attention mechanism, a copy mechanism and a coverage mechanism. In the online recommendation phase, for a given code snippet, we use the offline trained model to generate question titles to assist less experienced developers in writing questions more effectively. At the same time, we embed the given code snippet into a vector and retrieve the related questions with similar problematic code snippets.
BibTeX
@inproceedings{gao2021code,
  author = {Zhipeng Gao and Xin Xia and David Lo and John C. Grundy and Yuan-Fang Li},
  title = {{Code2Que: A Tool for Improving Question Titles from Mined Code Snippets in Stack Overflow}},
  booktitle = {ESEC/SIGSOFT FSE},
  year = {2021},
  doi = {10.1145/3468264.3473114},
}
Figure