Deep learning-based software engineering: progress, challenges, and opportunities

Xiangping Chen, Xing Hu, Yuan Huang, He Jiang, Weixing Ji, Yanjie Jiang, Yanyan Jiang, Bo Liu, Hui Liu, Xiaochen Li, Xiaoli Lian, Guozhu Meng, Xin Peng, Hailong Sun, Lin Shi, Bo Wang, Chong Wang, Jiayi Wang, Tiantian Wang, Jifeng Xuan, Xin Xia, Yibiao Yang, Yixin Yang, Li Zhang, Yuming Zhou, Lu Zhang

Abstract

Recent advances in deep learning (DL) have significantly impacted various research disciplines, including natural language processing and image processing. In software engineering (SE), DL techniques are increasingly employed to automate tasks such as code generation, refactoring, and fault localization. While existing surveys often focus on the underlying neural architectures, this study presents the first task-oriented survey. Covering twelve major SE subareas across the entire software lifecycle—from requirements engineering to developer collaboration—we analyze 601 papers to highlight major advances, available datasets, and future opportunities.

Introduction: A Task-Oriented Perspective

Deep learning has transitioned from a research curiosity to a fundamental driver of innovation in software engineering. However, most literature reviews categorize research by technique (e.g., "how to use a Transformer for code"). This paper addresses the need for a task-oriented perspective, focusing on how DL solves specific problems within the distinct phases of the software development lifecycle. By breaking down the "silos" between sub-disciplines, the authors provide a unified roadmap for the future of DL-based SE.

Methodology and Scope

The authors conducted a systematic analysis of 601 publications from premier SE and AI venues (including ICSE, ASE, NeurIPS, and AAAI) published between 2000 and 2023. These works were categorized into 12 primary software engineering tasks.

Distribution of surveyed papers across 12 software engineering tasks.

As shown in the table above, research effort is not distributed equally. Subareas like Bug finding (114 papers) and Program repair (64 papers) have received significant attention, while earlier lifecycle phases like Requirements engineering and Software refactoring represent emerging frontiers with fewer studies.

The Fuel of DL: Datasets and Benchmarks

The survey serves as a directory for critical benchmarks that enable DL research:

  • Code Intelligence: CodeXGLUE and CodeSearchNet act as the "ImageNet" for code, providing multi-language data for various tasks.
  • Bug/Repair: Defects4J and ManyBugs remain the gold standards for evaluating automated patching.
  • Clone Detection: BigCloneBench is essential for testing a model's ability to understand semantic logic beyond surface-level syntax.

A notable finding is the "data hunger" in Requirements Engineering, where datasets are often proprietary and significantly smaller than the massive repositories available for code search.

Key Insights: Beyond Vanilla Models

The shift from manually defined metrics (like cyclomatic complexity) to learned features is the most significant trend. Key takeaways include:

  1. Structural Awareness: Code is not just text; it is hierarchical. The most effective models incorporate Abstract Syntax Trees (ASTs) or Graph Neural Networks (GNNs) to capture control and data flow.
  2. Task Duality: Many SE tasks are mirrors of each other (e.g., Code Generation and Code Summarization). "Dual learning" frameworks allow models to improve by learning from these reciprocal relationships.
  3. Maintenance Dominance: The bulk of current research focuses on the maintenance phase, particularly bug detection and repair. This leaves a massive opportunity for DL applications in the design and requirements phases.

Results and the LLM Revolution

DL models have achieved unprecedented performance in tasks like Code Clone Detection, particularly in identifying "Type-4" clones (semantically similar but syntactically different). Furthermore, the rise of Large Language Models (LLMs) like ChatGPT and GitHub Copilot has moved automated code generation into the daily workflow of millions of developers. However, these models still face challenges with "hallucinations"—generating code that appears correct but contains logical flaws.

Challenges and Future Directions

Despite the progress, several hurdles remain:

  • Interpretability: Developers need to understand why a model flags a bug. Moving away from "black box" models toward interpretable AI is crucial for trust.
  • Generalization: Performance often degrades when models are applied to domain-specific or less popular programming languages.
  • AI-in-the-loop: The future lies in collaborative systems where AI acts as a partner that can explain its reasoning and adapt based on real-time human feedback.

Summary updated

Abstract

Researchers have recently achieved significant advances in deep learning techniques, which in turn has substantially advanced other research disciplines, such as natural language processing, image processing, speech recognition, and software engineering. Various deep learning techniques have been successfully employed to facilitate software engineering tasks, including code generation, software refactoring, and fault localization. Many studies have also been presented in top conferences and journals, demonstrating the applications of deep learning techniques in resolving various software engineering tasks. However, although several surveys have provided overall pictures of the application of deep learning techniques in software engineering, they focus more on learning techniques, that is, what kind of deep learning techniques are employed and how deep models are trained or fine-tuned for software engineering tasks. We still lack surveys explaining the advances of subareas in software engineering driven by deep learning techniques, as well as challenges and opportunities in each subarea. To this end, in this study, we present the first task-oriented survey on deep learning-based software engineering. It covers twelve major software engineering subareas significantly impacted by deep learning techniques. Such subareas spread out through the whole lifecycle of software development and maintenance, including requirements engineering, software development, testing, maintenance, and developer collaboration. As we believe that deep learning may provide an opportunity to revolutionize the whole discipline of software engineering, providing one survey covering as many subareas as possible in software engineering can help future research push forward the frontier of deep learning-based software engineering more systematically. For each of the selected subareas, we highlight the major advances achieved by applying deep learning techniques with pointers to the available datasets in such a subarea. We also discuss the challenges and opportunities concerning each of the surveyed software engineering subareas.
BibTeX
@article{chen2025deep,
  author = {Xiangping Chen and Xing Hu and Yuan Huang and He Jiang and Weixing Ji and Yanjie Jiang and Yanyan Jiang and Bo Liu and Hui Liu and Xiaochen Li and Xiaoli Lian and Guozhu Meng and Xin Peng and Hailong Sun and Lin Shi and Bo Wang and Chong Wang and Jiayi Wang and Tiantian Wang and Jifeng Xuan and Xin Xia and Yibiao Yang and Yixin Yang and Li Zhang and Yuming Zhou and Lu Zhang},
  title = {{Deep learning-based software engineering: progress, challenges, and opportunities}},
  journal = {Science China Information Sciences},
  year = {2025},
  url = {https://openreview.net/forum?id=45LymbNswV},
}
Figure