Revisiting Exception Handling Practices with Exception Flow Analysis

Guilherme B. de Pádua, Weiyi Shang

Abstract

Modern programming languages, such as Java and C#, typically provide features that handle exceptions. These features separate error-handling code from regular source code and aim to assist in the practice of software comprehension and maintenance. Having acknowledged the advantages of exception handling features, their misuse can still cause reliability degradation or even catastrophic software failures. Prior studies on exception handling aim to understand the practices of exception handling in its different components, such as the origin of the exceptions and the handling code of the exceptions. Yet, the observed findings were scattered and diverse. In this paper, to complement prior research findings on exception handling, we study its features by enriching the knowledge of handling code with a flow analysis of exceptions. Our case study is conducted with over 10K exception handling blocks, and over 77K related exception flows from 16 open-source Java and C# (.NET) libraries and applications. Our case study results show that each try block has up to 12 possible potentially recoverable yet propagated exceptions. More importantly, 22% of the distinct possible exceptions can be traced back to multiple methods (average of 1.39 and max of 34). Such results highlight the additional challenge of composing quality exception handling code. To make it worse, we confirm that there is a lack of documentation of the possible exceptions and their sources. However, such critical information can be identified by exception flow analysis on well-documented API calls (e.g., JRE and .NET documentation). Finally, we observe different strategies in exception handling code between Java and C#. Our findings highlight the opportunities of leveraging automated software analysis to assist in exception handling practices and signify the need of more further in-depth studies on exception handling practice.

Introduction: The Visibility Gap

Exception handling is designed to separate error-processing from regular application logic, theoretically making code more robust and maintainable. However, in practice, it often becomes a source of technical debt. Developers frequently encounter "anti-patterns"—such as swallowing exceptions or catching generic Exception types—which can lead to silent failures or catastrophic crashes.

The core challenge identified in this research is visibility. When a developer writes a try-catch block, they often lack precise knowledge of which exceptions might be thrown by the methods they are calling, especially when those methods invoke other methods several layers deep in the call stack. This paper bridges that gap by tracing the entire "flow" of an exception from its origin to its eventual destination.

Methodology: Exception Flow Analysis

To analyze these patterns, the authors developed an automated tool to perform Exception Flow Analysis across 16 major open-source projects (including Apache ANT, Guava, Hadoop, and SignalR). The process involves three main steps:

  1. AST Parsing: Building Abstract Syntax Trees using Eclipse JDT (Java) and Roslyn (C#).
  2. Call Graph Construction: Mapping method invocations to see how exceptions propagate up the stack.
  3. Source Identification: Identifying four distinct sources of exceptions: explicit throw statements, method declarations, in-code comments (e.g., Javadoc), and external documentation (official JRE and .NET documentation).
Exception flow and sources identified by the analysis tool.

The Documentation Gap

A primary finding of the study is that developers are essentially "flying blind." The analysis revealed that 93% of possible exceptions in Java and 71% in C# can only be identified by consulting external API documentation rather than the source code itself. This lack of internal visibility explains why developers often struggle to write specific, effective handlers.

Venn diagram showing that most Java unchecked exceptions are only documented externally.

Key Insights and Results

Complexity and Multiple Origins

The study found that exception handling is significantly more complex than previously thought. The median number of distinct possible exceptions per try block is 4 for C# and 2 for Java, with some complex blocks containing up to 33 distinct exceptions. Furthermore, 22% of exceptions in a single try block can be traced back to multiple different method calls. For instance, one block in the Umbraco project contained a System.ArgumentNullException that could originate from 14 different locations, making precise recovery extremely difficult.

The "Recoverable Leak"

A significant number of exceptions documented as "recoverable" are allowed to propagate (bubble up) without being handled. In Java, over 19% of try blocks propagate at least one potentially recoverable exception. This suggests that even when recovery is possible, the complexity of identifying the source prevents developers from implementing it.

Handling Strategies: Subsumption vs. Specificity

Most developers favor "Subsumption" (catching a parent class like IOException) over "Specific" handling. While easier to write, this prevents tailored recovery actions. The study also noted language-specific differences: Java developers are more likely to use "Throw Wrap" (re-throwing) or leave "TODO" comments, likely due to the friction of Java's checked exceptions requirement.

Conclusion and Future Directions

The authors acknowledge that static analysis may under-estimate flows due to polymorphism or dynamic loading. However, the results clearly signify a need for better tooling. Future IDEs could incorporate real-time flow analysis to suggest exactly which exceptions to catch based on deep-stack library analysis, potentially reducing the industry's reliance on generic, fragile "catch-all" blocks.

Summary updated

Abstract

Modern programming languages, such as Java and C#, typically provide features that handle exceptions. These features separate error-handling code from regular source code and aim to assist in the practice of software comprehension and maintenance. Having acknowledged the advantages of exception handling features, their misuse can still cause reliability degradation or even catastrophic software failures. Prior studies on exception handling aim to understand the practices of exception handling in its different components, such as the origin of the exceptions and the handling code of the exceptions. Yet, the observed findings were scattered and diverse. In this paper, to complement prior research findings on exception handling, we study its features by enriching the knowledge of handling code with a flow analysis of exceptions. Our case study is conducted with over 10K exception handling blocks, and over 77K related exception flows from 16 open-source Java and C# (.NET) libraries and applications. Our case study results show that each try block has up to 12 possible potentially recoverable yet propagated exceptions. More importantly, 22% of the distinct possible exceptions can be traced back to multiple methods (average of 1.39 and max of 34). Such results highlight the additional challenge of composing quality exception handling code. To make it worse, we confirm that there is a lack of documentation of the possible exceptions and their sources. However, such critical information can be identified by exception flow analysis on well- documented API calls (e.g., JRE and .NET documentation). Finally, we observe different strategies in exception handling code between Java and C#. Our findings highlight the opportunities of leveraging automated software analysis to assist in exception handling practices and signify the need of more further in-depth studies on exception handling practice.
BibTeX
@inproceedings{padua2017revisiting,
  author = {Guilherme B. de Pádua and Weiyi Shang},
  title = {{Revisiting Exception Handling Practices with Exception Flow Analysis}},
  booktitle = {SCAM},
  year = {2017},
  url = {https://openreview.net/forum?id=2Jk6oabrGr},
}

Loading related content...

Figure