This paper, presented at the 2017 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), provides a deep dive into how developers actually handle (or mishandle) exceptions in the wild. By analyzing over 77,000 exception flows across major Java and C# projects, the authors reveal a significant gap between language features and real-world implementation.
TL;DR
Researchers used automated "exception flow analysis" to track how exceptions travel from their source to their handlers across 16 major open-source projects. They discovered that developers frequently propagate potentially recoverable errors and rely heavily on "catch-all" strategies because they lack clear documentation on what exceptions their code might actually throw.
What Problem Does This Paper Address?
Exception handling is supposed to separate error-processing from regular logic, making code more robust. However, in practice, it’s often a source of technical debt. Developers frequently encounter "anti-patterns"—like swallowing exceptions or catching generic Exception types—which can lead to silent failures or catastrophic crashes.
The core challenge is visibility. When a developer writes a try-catch block, they often don't know exactly which exceptions might be thrown by the methods they are calling, especially when those methods call other methods several layers deep. Prior research looked at exception handling in fragments; this paper aims to bridge the gap by tracing the entire "flow" of an exception from its origin to its eventual destination.
Methods
The authors developed an automated tool to perform Exception Flow Analysis. The process involves three main steps:
- AST Parsing: Using Eclipse JDT (for Java) and Roslyn (for C#) to build Abstract Syntax Trees of the source code.
- Call Graph Construction: The tool builds a map of method invocations to see how exceptions propagate up the stack.
- Source Identification: The tool identifies four sources of exceptions:
- Explicit
throwstatements. - Method declarations (the
throwsclause in Java). - In-code comments (like Javadoc).
- External Documentation: Crucially, the tool parses official JRE and .NET documentation to identify exceptions thrown by standard library calls that aren't explicitly declared in the source code.
Datasets
The study conducted a case study on 16 open-source projects (7 in C# and 9 in Java), including heavyweights like:
- Java: Apache ANT, Guava, Hadoop (Common, HDFS, MapReduce, YARN), and the Spring Framework.
- C#: Glimpse, ShareX, SharpDevelop, and SignalR.
The total dataset comprised approximately 10,110 try blocks, 11,797 catch blocks, and 77,000+ exception flows across 3.8 million lines of code (KLOC).
Key Insights and Ideas
- The "Recoverable Leak": A significant number of exceptions that are officially documented as "recoverable" are simply allowed to propagate (bubble up) without being handled. In Java, over 19% of try blocks propagate at least one potentially recoverable exception.
- The Documentation Gap: Developers are flying blind. The study found that 93% of possible exceptions in Java and 71% in C# can only be identified by looking at external API documentation, rather than the source code itself.
- Multiple Origins: 22% of exceptions in a single
tryblock can be traced back to multiple different method calls. For example, a single block in theUmbracoproject had aSystem.ArgumentNullExceptionthat could originate from 14 different places. This makes precise handling extremely difficult for a human developer. - Subsumption Over Specificity: Most developers use "Subsumption" (catching a parent class like
IOException) rather than "Specific" handling. While easier to write, this prevents the program from taking tailored recovery actions based on the specific error. - Language Differences: Java developers are more likely to use "Throw Wrap" (re-throwing an exception) or leave "TODO" comments in catch blocks compared to C# developers, possibly due to the friction caused by Java's "checked exceptions" requirement.
Key Results
- Complexity: The median number of distinct possible exceptions per try block is 4 for C# and 2 for Java. Some complex blocks had up to 33 distinct exceptions.
- Propagation: Each try block has, on average, up to 12 possible exceptions that are propagated rather than handled.
- Handling Actions: "Method" calls (executing a specific method to handle the error) and "Throw Wrap" are the most common actions. In Java, "Log" actions are significantly more common when using specific handlers versus generic ones.
Limitations and Future Work
The authors acknowledge that their static analysis approach might suffer from "under-estimation." Because they use call graph approximations, they might miss some exceptions triggered by polymorphism or dynamic method loading. Furthermore, the study assumes that external documentation is accurate and up-to-date, which isn't always the case.
Future Work: The authors suggest that IDEs should incorporate this type of flow analysis in real-time. Imagine an editor that suggests exactly which exceptions to catch based on a deep-stack analysis of the libraries you are using, potentially reducing the reliance on generic "catch-all" blocks.
Summary updated