Automatic fault localization for client-side JavaScript

Frolin S. Ocariza Jr., Guanpeng Li, Karthik Pattabiraman, Ali MesbahSoftw. Test. Verification Reliab 2016 · DOI

TL;DR

This paper introduces AUTOFLOX, an automated technique for pinpointing the root cause of client-side JavaScript errors, particularly those related to Document Object Model (DOM) interactions. By leveraging dynamic analysis and backward slicing, AUTOFLOX achieves high accuracy in localizing these complex bugs, even in minified or dynamically generated code, significantly reducing debugging time for web developers.

What Problem Does This Paper Address?

Modern web applications heavily rely on client-side JavaScript to enhance interactivity and user experience. However, this reliance introduces significant complexity, making JavaScript code prone to errors. Debugging these applications is notoriously difficult due to JavaScript's dynamic, loosely typed, and asynchronous nature, coupled with its intricate interactions with the DOM. A substantial portion of these bugs (over 65%) are "DOM-related," meaning they stem from or manifest through interactions with the web page's structure. Manually locating the source of these errors is a time-consuming and tedious process for developers. Existing research has largely overlooked automated fault localization for client-side JavaScript, leaving a critical gap in the developer toolchain.

Methods

The core of AUTOFLOX is a two-phase approach:

  1. Trace Collection: The system instruments client-side JavaScript code on-the-fly. As the web application runs (either through crawling or user interaction), it records a detailed trace of executed statements, including function names, line numbers, variable scopes, and their values. Special markers are added to the trace to denote failures (FAILURE), asynchronous calls (ASYNCCALL), and the start of asynchronous function executions (ASYNC). This instrumentation is designed to handle challenging JavaScript features like eval, anonymous functions, and minified code.
  2. Trace Analysis: Once a failure occurs, the collected trace is analyzed. The system first partitions the trace into sequences corresponding to specific events. It then identifies the "relevant sequence" that led to the failure, often by prepending sequences related to asynchronous calls. The key step is then to perform a dynamic backward slice from the failing line to identify the "direct DOM access" – the specific DOM API call or property access that directly caused the error (e.g., returning null or undefined). This backward slicing is guided by the propagation of null values, effectively tracing the error back to its origin.

Datasets

The paper evaluates AUTOFLOX on:

  • Six open-source web applications: These applications were used for fault injection experiments to measure accuracy.
  • Three production websites: These were used to assess robustness in real-world scenarios.
  • Seven other web applications: For these, AUTOFLOX was tested on 20 actual, reported bugs.

The authors note that for the fault injection experiments, they manually injected faults by mutating parameters of DOM access functions to ensure they returned null.

Key Insights and Ideas

  • DOM interactions are a major source of JavaScript bugs: The paper strongly emphasizes that a significant majority of client-side JavaScript bugs are related to DOM interactions, making them a critical target for automated tools.
  • JavaScript's dynamic nature complicates fault localization: Features like eval, anonymous functions, and minified code make static analysis difficult and manual debugging error-prone. AUTOFLOX's ability to handle these features is a key differentiator.
  • Backward slicing is effective for DOM-related faults: By tracing the propagation of null values backward from the point of failure, the system can effectively pinpoint the direct DOM interaction that initiated the error.
  • No false positives is a significant achievement: The reported results show AUTOFLOX achieving zero false positives, meaning it never incorrectly identifies a fault location.

Key Results

  • High Accuracy: AUTOFLOX achieved an overall accuracy of 96.4% in localizing injected DOM-related faults across six open-source web applications.
  • No False Positives: The tool demonstrated zero false positives in all experiments.
  • Improved Accuracy with Feature Support: Enabling support for eval, anonymous functions, and minified code significantly boosted accuracy from 77.1% to 96.4%.
  • Real-world Bug Localization: AUTOFLOX successfully identified the direct DOM access for all 20 real bugs tested from seven different web applications.
  • Performance: While trace collection can incur overhead (ranging from 30.9% to 119.3% on production sites), the trace analysis phase is very fast (averaging 0.1 seconds). The overhead is considered acceptable as trace collection is only active when a fault needs localization.

Limitations and Future Work

  • Manual Element Specification: Currently, AUTOFLOX requires users to specify which DOM elements to interact with to trigger a failure. This can be tedious if the user isn't familiar with the application's DOM structure. The Eclipse plugin offers some automation, but further work is needed to fully automate this process.
  • setTimeout with Function Literals: The current implementation does not fully support cases where function literals are passed as parameters to functions like setTimeout. This is considered an implementation issue that can be addressed.
  • Code-terminating Faults Focus: The current approach primarily targets "code-terminating" faults (those causing exceptions). Future work aims to extend the technique to more automatically handle "output DOM-related faults" (those that cause incorrect DOM output without halting execution).
  • User Studies: The authors plan to conduct user studies to evaluate the ease of use and overall efficacy of AUTOFLOX in practice.

Summary updated

BibTeX
@article{jr2016automatic,
  author = {Frolin S. Ocariza Jr. and Guanpeng Li and Karthik Pattabiraman and Ali Mesbah},
  title = {{Automatic fault localization for client-side JavaScript}},
  journal = {Softw. Test. Verification Reliab.},
  year = {2016},
  doi = {10.1002/stvr.1576},
}
Figure