BERT-Based Code Learning for Exception Localization and Type Prediction

Chongyu Zhang, Qiping Tao, Liangyu Chen, Min ZhangAAAI 2025 · DOI

TL;DR

This paper introduces CodeHunter, a novel AI-powered system that automatically identifies lines of code likely to cause exceptions and predicts the type of exception that might occur. By leveraging BERT for code understanding and Bi-LSTM for sequence analysis, CodeHunter significantly outperforms existing methods in both exception localization and type prediction, offering a valuable tool for improving software reliability.

What Problem Does This Paper Address?

Exception handling is a critical but often challenging aspect of software development. Ensuring that programs can gracefully manage unexpected situations is vital for system stability and security. Traditional approaches rely heavily on programmer expertise, which can lead to oversights, inconsistent handling, and incomplete exception management. While AI has shown promise in this area, existing methods often struggle to fully capture the contextual nuances of code surrounding potential exceptions, limiting their effectiveness. This paper aims to bridge this gap by developing a more comprehensive AI model for both pinpointing where exceptions might occur (localization) and predicting what kind of exception it will be (type prediction).

Methods

The core of CodeHunter lies in its two-stage approach:

  1. Exception Localization:
    • Data Preprocessing: Code is enriched using Abstract Syntax Tree (AST) parsing to capture syntax features and variable name substitution to retain class information.
  2. Feature Representation: A BERT-based model (specifically CodeT5+) is used to generate rich feature representations for each line of code.
  3. Sequence Labeling: A Bi-LSTM network acts as a sequence labeler, processing the BERT embeddings to predict whether each line of code is likely to raise an exception (labeled 0 or 1).
  4. Exception Type Prediction:
    • Data Preprocessing: Similar to localization, AST parsing and tokenization are employed. Crucially, code is divided into two parts: code before the try block and code within the try block.
  5. Feature Representation: Two separate BERT-based models process the "before try" and "in try" code segments. A self-attention mechanism is used to learn weights, giving higher importance to the code within the try block.
  6. Sequence Labeling: Two independent Bi-LSTM models process the outputs from the BERT models for each code segment.
  7. Output Layer: A fully connected layer followed by a Softmax activation function predicts the probability distribution over possible exception types, with the highest probability indicating the predicted type.

Datasets

The paper evaluates CodeHunter on three publicly available datasets:

  • Nexgen Dataset: Contains 377,923 methods from 2,000 popular Java projects. Used for both localization and type prediction.
  • D-REX Dataset: Comprises 442,446 methods from 200,000 Java projects. This dataset contains code tokens but not source code, making it suitable only for exception type prediction.
  • Neurex Dataset: Includes 307,646 methods from 5,726 highly-rated Java projects. Used for both localization and type prediction.

Key Insights and Ideas

  • Context is King: The paper strongly emphasizes that understanding the context around a potential exception is as important as the code itself. This is why separating code before and within try blocks and using attention mechanisms to weigh their importance is a key innovation.
  • AST and Variable Names Matter: The experiments demonstrate that incorporating AST parsing and variable name substitution significantly boosts performance, especially for exception localization. This suggests that structural and semantic information beyond raw code tokens is crucial.
  • BERT's Power for Code: The success of BERT-based models in capturing complex code semantics is a recurring theme, enabling better feature extraction for downstream tasks.
  • Bi-LSTM for Sequence Understanding: Bi-LSTM proves effective in processing sequential code data and identifying patterns indicative of exceptions or predicting their types.
  • Practical Utility: The model is implemented as an IDE plugin, highlighting a focus on real-world developer utility.

Key Results

  • Exception Localization:
    • On the Nexgen dataset, CodeHunter achieved 79.7% accuracy, outperforming baselines by up to 11.2%.
  • On the Neurex dataset, CodeHunter achieved an impressive 98.6% accuracy, significantly surpassing the Neurex baseline (87.4%).
  • Exception Type Prediction:
    • On the D-REX dataset, CodeHunter achieved a Top-1 accuracy of 87.26%, outperforming M-E-F (70.95%) and D-REX (81.09%).
  • On the Nexgen dataset, CodeHunter achieved a Top-1 accuracy of 93.86%, significantly outperforming M-E-F and a variant without AST.
  • On the Neurex dataset, CodeHunter achieved 92.0% accuracy, outperforming the Neurex baseline (73.3%).

Limitations and Future Work

  • Dataset Specificity: The paper notes that while CodeHunter performs well, precision on the Neurex dataset was slightly lower than a baseline, potentially due to differences in training set construction. This suggests that dataset characteristics can influence performance.
  • GPT-3.5 Performance: The authors mention that initial experiments with GPT-3.5 for exception prediction yielded low accuracy (24.08%), indicating that while LLMs are powerful, specific fine-tuning and architectural choices are still critical for specialized tasks like this.
  • Future Work: The paper's conclusion implies that the IDE plugin implementation is a step towards practical application, suggesting further integration and refinement for developer workflows. The success of AST and variable name substitution also points towards exploring other code representation enhancement techniques.

Summary updated

BibTeX
@inproceedings{zhang2025bert,
  author = {Chongyu Zhang and Qiping Tao and Liangyu Chen and Min Zhang},
  title = {{BERT-Based Code Learning for Exception Localization and Type Prediction}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i1.32090},
}
Figure