Revisiting Exception Handling Practices with Exception Flow Analysis

Guilherme B. de Pádua, Weiyi Shang

This paper, presented at the 2017 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), provides a deep dive into how developers actually handle (or mishandle) exceptions in the wild. By analyzing over 77,000 exception flows across major Java and C# projects, the authors reveal a significant gap between language features and real-world implementation.

TL;DR

Researchers used automated "exception flow analysis" to track how exceptions travel from their source to their handlers across 16 major open-source projects. They discovered that developers frequently propagate potentially recoverable errors and rely heavily on "catch-all" strategies because they lack clear documentation on what exceptions their code might actually throw.

What Problem Does This Paper Address?

Exception handling is supposed to separate error-processing from regular logic, making code more robust. However, in practice, it’s often a source of technical debt. Developers frequently encounter "anti-patterns"—like swallowing exceptions or catching generic Exception types—which can lead to silent failures or catastrophic crashes.

The core challenge is visibility. When a developer writes a try-catch block, they often don't know exactly which exceptions might be thrown by the methods they are calling, especially when those methods call other methods several layers deep. Prior research looked at exception handling in fragments; this paper aims to bridge the gap by tracing the entire "flow" of an exception from its origin to its eventual destination.

Methods

The authors developed an automated tool to perform Exception Flow Analysis. The process involves three main steps:

  1. AST Parsing: Using Eclipse JDT (for Java) and Roslyn (for C#) to build Abstract Syntax Trees of the source code.
  2. Call Graph Construction: The tool builds a map of method invocations to see how exceptions propagate up the stack.
  3. Source Identification: The tool identifies four sources of exceptions:
  4. Explicit throw statements.
  5. Method declarations (the throws clause in Java).
  6. In-code comments (like Javadoc).
  7. External Documentation: Crucially, the tool parses official JRE and .NET documentation to identify exceptions thrown by standard library calls that aren't explicitly declared in the source code.

Datasets

The study conducted a case study on 16 open-source projects (7 in C# and 9 in Java), including heavyweights like:

  • Java: Apache ANT, Guava, Hadoop (Common, HDFS, MapReduce, YARN), and the Spring Framework.
  • C#: Glimpse, ShareX, SharpDevelop, and SignalR.

The total dataset comprised approximately 10,110 try blocks, 11,797 catch blocks, and 77,000+ exception flows across 3.8 million lines of code (KLOC).

Key Insights and Ideas

  • The "Recoverable Leak": A significant number of exceptions that are officially documented as "recoverable" are simply allowed to propagate (bubble up) without being handled. In Java, over 19% of try blocks propagate at least one potentially recoverable exception.
  • The Documentation Gap: Developers are flying blind. The study found that 93% of possible exceptions in Java and 71% in C# can only be identified by looking at external API documentation, rather than the source code itself.
  • Multiple Origins: 22% of exceptions in a single try block can be traced back to multiple different method calls. For example, a single block in the Umbraco project had a System.ArgumentNullException that could originate from 14 different places. This makes precise handling extremely difficult for a human developer.
  • Subsumption Over Specificity: Most developers use "Subsumption" (catching a parent class like IOException) rather than "Specific" handling. While easier to write, this prevents the program from taking tailored recovery actions based on the specific error.
  • Language Differences: Java developers are more likely to use "Throw Wrap" (re-throwing an exception) or leave "TODO" comments in catch blocks compared to C# developers, possibly due to the friction caused by Java's "checked exceptions" requirement.

Key Results

  • Complexity: The median number of distinct possible exceptions per try block is 4 for C# and 2 for Java. Some complex blocks had up to 33 distinct exceptions.
  • Propagation: Each try block has, on average, up to 12 possible exceptions that are propagated rather than handled.
  • Handling Actions: "Method" calls (executing a specific method to handle the error) and "Throw Wrap" are the most common actions. In Java, "Log" actions are significantly more common when using specific handlers versus generic ones.

Limitations and Future Work

The authors acknowledge that their static analysis approach might suffer from "under-estimation." Because they use call graph approximations, they might miss some exceptions triggered by polymorphism or dynamic method loading. Furthermore, the study assumes that external documentation is accurate and up-to-date, which isn't always the case.

Future Work: The authors suggest that IDEs should incorporate this type of flow analysis in real-time. Imagine an editor that suggests exactly which exceptions to catch based on a deep-stack analysis of the libraries you are using, potentially reducing the reliance on generic "catch-all" blocks.

Summary updated

Abstract

Modern programming languages, such as Java and C#, typically provide features that handle exceptions. These features separate error-handling code from regular source code and aim to assist in the practice of software comprehension and maintenance. Having acknowledged the advantages of exception handling features, their misuse can still cause reliability degradation or even catastrophic software failures. Prior studies on exception handling aim to understand the practices of exception handling in its different components, such as the origin of the exceptions and the handling code of the exceptions. Yet, the observed findings were scattered and diverse. In this paper, to complement prior research findings on exception handling, we study its features by enriching the knowledge of handling code with a flow analysis of exceptions. Our case study is conducted with over 10K exception handling blocks, and over 77K related exception flows from 16 open-source Java and C# (.NET) libraries and applications. Our case study results show that each try block has up to 12 possible potentially recoverable yet propagated exceptions. More importantly, 22% of the distinct possible exceptions can be traced back to multiple methods (average of 1.39 and max of 34). Such results highlight the additional challenge of composing quality exception handling code. To make it worse, we confirm that there is a lack of documentation of the possible exceptions and their sources. However, such critical information can be identified by exception flow analysis on well-documented API calls (e.g., JRE and.NET documentation). Finally, we observe different strategies in exception handling code between Java and C#. Our findings highlight the opportunities of leveraging automated software analysis to assist in exception handling practices and signify the need of more further in-depth studies on exception handling practice.
BibTeX
@inproceedings{padua2017revisiting,
  author = {Guilherme B. de Pádua and Weiyi Shang},
  title = {{Revisiting Exception Handling Practices with Exception Flow Analysis}},
  booktitle = {SCAM},
  year = {2017},
  url = {https://openreview.net/forum?id=2Jk6oabrGr},
}

Loading related content...

Figure