iFixR: Bug Report driven Program Repair

Anil Koyuncu, Kui Liu, Tegawendé F. Bissyandé, Dongsun Kim, Martin Monperrus, Jacques Klein, Yves Le Traon

Abstract

Issue tracking systems are essential in modern software development for collecting feedback. A key goal of software maintenance is systematizing patch generation for user-reported bugs. While automated program repair (APR) has gained momentum, most research focuses on "generate-and-validate" setups driven by well-defined test suites. However, the assumption that relevant test cases exist often fails in practice; many bugs are reported before a revealing test suite is available. We investigate iFixR, a repair pipeline driven by bug reports: (1) bug reports are fed to an IR-based fault localizer; (2) patches are generated from fix patterns and validated via regression testing; (3) a prioritized list of patches is proposed to developers. Evaluating iFixR on an enriched Defects4J dataset, we found it generates genuine/plausible patches for 21/44 faults, accurately placing them in the top-5 recommendations for 8/13 of these cases without relying on future test cases.

The Reality Gap in Automated Program Repair

Most APR tools operate in a loop where a failing test case identifies the bug (Fault Localization) and passing tests ensure the fix doesn't cause regressions (Validation). However, this creates a significant gap between research and real-world application. In practice, when a bug is reported, a reproducing test case rarely exists.

Analysis of failing test cases in the Defects4J dataset, highlighting the scarcity of pre-existing failing tests.

An analysis of the Defects4J benchmark reveals a startling reality: for approximately 92% of bugs (365 out of 395), no failing test case exists at the time of the report. Traditional APR tools are effectively "blind" in these scenarios, as they rely on information that only becomes available after a developer has already manually triaged the bug and written a reproducing test.

The iFixR Pipeline

iFixR introduces a workflow designed to mimic the natural workflow of a human developer, moving from a natural language report to a validated code fix.

The iFixR repair pipeline, showing the transition from bug reports to prioritized patch candidates.
  1. IR-based Fault Localization (IRFL): Instead of executing code to find failures, iFixR treats the bug report as a search query. It uses Information Retrieval (IR) techniques to identify code statements semantically similar to the report text.
  2. Patch Generation: The tool utilizes a library of "fix patterns"—templates derived from common human-written fixes—to mutate the suspicious code identified in the previous step.
  3. Regression-only Validation: Assuming no failing test is available, iFixR validates patches against the existing regression test suite. A patch is considered a candidate if it passes all current tests without introducing new failures.
  4. Prioritization Heuristics: To assist developers, candidates are ranked based on minimal AST changes, IR similarity scores (suspiciousness), and the nature of the affected code elements.

Key Insights and Results

Reducing Overfitting

Traditional APR tools often "overfit," creating patches that satisfy a single failing test but fail to address the underlying logic. iFixR's IR-based localization anchors the fix to the semantic description of the bug, which the authors found leads to more robust, less overfitted results.

Performance and Efficiency

  • Repair Success: iFixR successfully generated genuine or plausible patches for 21 out of 44 Defects4J faults using its IR-based localizer.
  • Ranking Accuracy: For the faults it could fix, iFixR placed a correct or plausible patch in the top-5 recommendations in 61% of cases (8 out of 13).
  • Time-Saving: On average, iFixR could have localized bugs 26 days before the first reproducing test case was submitted to the project repositories, significantly accelerating the maintenance lifecycle.

Conclusion

iFixR demonstrates that bug reports are a viable and realistic driver for automated program repair. By shifting away from a dependency on failing test cases, iFixR aligns APR research with actual development practices, offering a tool that can suggest fixes nearly a month before traditional methods could even begin their work. Future work involves integrating iFixR with automated test generation to provide an even stronger oracle for correctness.

Summary updated

Abstract

Issue tracking systems are commonly used in modern software development for collecting feedback from users and developers. An ultimate automation target of software maintenance is then the systematization of patch generation for user-reported bugs. Although this ambition is aligned with the momentum of automated program repair, the literature has, so far, mostly focused on generate-and-validate setups where fault localization and patch generation are driven by a well-defined test suite. On the one hand, however, the common (yet strong) assumption on the existence of relevant test cases does not hold in practice for most development settings: many bugs are reported without the available test suite being able to reveal them. On the other hand, for many projects, the number of bug reports generally outstrips the resources available to triage them. Towards increasing the adoption of patch generation tools by practitioners, we investigate a new repair pipeline, iFixR, driven by bug reports: (1) bug reports are fed to an IR-based fault localizer; (2) patches are generated from fix patterns and validated via regression testing; (3) a prioritized list of generated patches is proposed to developers. We evaluate iFixR on the Defects4J dataset, which we enriched (i.e., faults are linked to bug reports) and carefully-reorganized (i.e., the timeline of test-cases is naturally split). iFixR generates genuine/plausible patches for 21/44 Defects4J faults with its IR-based fault localizer. iFixR accurately places a genuine/plausible patch among its top-5 recommendation for 8/13 of these faults (without using future test cases in generation-and-validation).
BibTeX
@inproceedings{koyuncu2019ifixr,
  author = {Anil Koyuncu and Kui Liu and Tegawendé F. Bissyandé and Dongsun Kim and Martin Monperrus and Jacques Klein and Yves Le Traon},
  title = {{iFixR: Bug Report driven Program Repair}},
  booktitle = {ESEC/SIGSOFT FSE},
  year = {2019},
  url = {https://openreview.net/forum?id=HbNV2Z62Ql5},
}

Loading related content...

Figure