Auto-completing bug reports for Android applications

Kevin Moran, Mario Linares Vásquez, Carlos Bernal-Cárdenas, Denys Poshyvanyk

TL;DR

FUSION is a novel bug-reporting framework for Android that bridges the "lexical gap" between end-users and developers. By using a combination of static and dynamic analysis to pre-map an application's interface, FUSION allows reporters to "auto-complete" reproduction steps, resulting in highly structured, screenshot-augmented reports that are significantly easier for developers to reproduce and fix.

What Problem Does This Paper Address?

There is a fundamental disconnect in software maintenance: bug reporters (users/testers) usually have functional knowledge of an app, while developers have intimate code-level knowledge. This "lexical gap" leads to bug reports written in vague natural language that lack the specific steps or technical context needed for a fix.

In the mobile world, this is exacerbated by complex, event-driven GUIs. Traditional trackers like Bugzilla or GitHub rely on unstructured text, leading to a high rate of non-reproducible bugs. The authors cite that only about 50% of developers believe bug reports are complete, and providing reproduction steps is often the most difficult task for a reporter.

Methods

The authors introduce FUSION, which operates on an Analyze → Generate paradigm:

  1. Analysis Phase (Pre-release):
    • Static Analysis (The "Primer"): Uses tools like apktool and dex2jar to extract GUI components, their types (buttons, spinners), and the specific Java classes/activities where they reside.
  2. Dynamic Analysis (The "Engine"): Employs a systematic depth-first search (DFS) using the UIAutomator framework. It "rips" the app to discover possible transitions, capturing screenshots and the precise screen coordinates of every interactive element.
  3. Report Generation Phase (Post-bug):
    • When a user encounters a bug, they use a web-based portal. As they begin typing a reproduction step, FUSION’s Auto-Completion Engine queries the pre-built database.
  4. It suggests the most likely next actions and components based on the app’s state model.
  5. The reporter selects from these suggestions and confirms the state via "in-situ" screenshots. The final report is a structured document containing precise steps, class names, and a visual gallery of the reproduction path.

Datasets

The researchers evaluated FUSION using:

  • 14 Open-Source Android Apps: Sourced from F-Droid, including popular tools like GnuCash, OI Notepad, and A Time Tracker.
  • 15 Real-World Bugs: Manually selected based on reproducibility and varying complexity (requiring at least three steps to manifest).
  • User Study Participants: 28 total participants (a mix of undergraduate "non-experts" and graduate "experts") were recruited to report and reproduce bugs using both FUSION and the Google Code Issue Tracker (GCIT).

Key Insights and Ideas

  • The Power of Constraints: By forcing users to select from a list of known GUI components rather than typing free-form text, FUSION eliminates ambiguity. Even if a user doesn't know the technical name of a widget, they can identify it via the suggested screenshots.
  • Bridging the Expertise Gap: One of the most striking findings was that FUSION effectively "leveled up" inexperienced reporters. While non-experts usually write superficial reports in traditional systems, FUSION's structured workflow forced them to provide the detailed information developers actually need.
  • Actionable Metadata: Because FUSION links GUI elements to source code during the analysis phase, the resulting bug reports automatically include the specific Java classes involved, essentially performing a "pre-localization" of the fault for the developer.

Key Results

  • Reproducibility: Reports generated with FUSION were significantly more reliable. Only 13 out of 120 FUSION reports were non-reproducible, compared to 23 out of 120 for the traditional GCIT system.
  • Auto-completion Accuracy: In the user study, FUSION was able to successfully auto-suggest every single reproduction step (without the user needing to manually enter "missing" data) in 11 out of 60 reports.
  • User Preference: While FUSION reports took slightly longer to create (due to the added detail), experienced developers overwhelmingly preferred them. On a usability scale, participants generally rated FUSION higher than original bug reports for ease of reproduction.
  • Efficiency for Non-Experts: Inexperienced reporters using FUSION produced reports that were nearly as easy to reproduce as those written by experts.

Limitations and Future Work

  • Gesture Support: The current DFS engine primarily handles click/tap actions. It does not yet automatically capture complex gestures like long-presses or multi-touch swipes, though users can manually enter these.
  • Context Sensitivity: FUSION struggles with bugs that depend on external sensors or state changes (e.g., GPS location, changing network signal, or device orientation), as these aren't always captured in the static/dynamic GUI map.
  • DFS Exhaustion: The dynamic "ripping" process is not exhaustive. If a bug exists in a deep, dynamically generated menu that the DFS missed, the auto-completion will fail, forcing the user to fall back to manual entry.
  • Future Directions: The authors plan to expand the engine to support more gestures and explore using the structured data for automatic fault localization and even automated bug fixing.

Summary updated

Abstract

The modern software development landscape has seen a shift in focus toward mobile applications as tablets and smartphones near ubiquitous adoption. Due to this trend, the complexity of these “apps” has been increasing, making development and maintenance challenging. Additionally, current bug tracking systems are not able to effectively support construction of reports with actionable information that directly lead to a bug’s resolution. To address the need for an improved reporting system, we introduce a novel solution, called FUSION, that helps users auto-complete reproduction steps in bug reports for mobile apps. FUSION links user-provided information to program artifacts extracted through static and dynamic analysis performed before testing or release. The approach that FUSION employs is generalizable to other current mobile software platforms, and constitutes a new method by which off-device bug reporting can be conducted for mobile software projects. In a study involving 28 participants we applied FUSION to support the maintenance tasks of reporting and reproducing defects from 15 real-world bugs found in 14 open source Android apps while qualitatively and qualitatively measuring the user experience of the system. Our results demonstrate that FUSION both effectively facilitates reporting and allows for more reliable reproduction of bugs from reports compared to traditional issue tracking systems by presenting more detailed contextual app information.
BibTeX
@inproceedings{moran2015auto,
  author = {Kevin Moran and Mario Linares Vásquez and Carlos Bernal-Cárdenas and Denys Poshyvanyk},
  title = {{Auto-completing bug reports for Android applications}},
  booktitle = {ESEC/SIGSOFT FSE},
  year = {2015},
  url = {https://openreview.net/forum?id=rBBqnugSZW5},
}

Loading related content...

Figure