Abstract
Issue tracking systems are essential in modern software development for collecting feedback. A key goal of software maintenance is systematizing patch generation for user-reported bugs. While automated program repair (APR) has gained momentum, most research focuses on "generate-and-validate" setups driven by well-defined test suites. However, the assumption that relevant test cases exist often fails in practice; many bugs are reported before a revealing test suite is available. We investigate iFixR, a repair pipeline driven by bug reports: (1) bug reports are fed to an IR-based fault localizer; (2) patches are generated from fix patterns and validated via regression testing; (3) a prioritized list of patches is proposed to developers. Evaluating iFixR on an enriched Defects4J dataset, we found it generates genuine/plausible patches for 21/44 faults, accurately placing them in the top-5 recommendations for 8/13 of these cases without relying on future test cases.
The Reality Gap in Automated Program Repair
Most APR tools operate in a loop where a failing test case identifies the bug (Fault Localization) and passing tests ensure the fix doesn't cause regressions (Validation). However, this creates a significant gap between research and real-world application. In practice, when a bug is reported, a reproducing test case rarely exists.
An analysis of the Defects4J benchmark reveals a startling reality: for approximately 92% of bugs (365 out of 395), no failing test case exists at the time of the report. Traditional APR tools are effectively "blind" in these scenarios, as they rely on information that only becomes available after a developer has already manually triaged the bug and written a reproducing test.
The iFixR Pipeline
iFixR introduces a workflow designed to mimic the natural workflow of a human developer, moving from a natural language report to a validated code fix.
- IR-based Fault Localization (IRFL): Instead of executing code to find failures, iFixR treats the bug report as a search query. It uses Information Retrieval (IR) techniques to identify code statements semantically similar to the report text.
- Patch Generation: The tool utilizes a library of "fix patterns"—templates derived from common human-written fixes—to mutate the suspicious code identified in the previous step.
- Regression-only Validation: Assuming no failing test is available, iFixR validates patches against the existing regression test suite. A patch is considered a candidate if it passes all current tests without introducing new failures.
- Prioritization Heuristics: To assist developers, candidates are ranked based on minimal AST changes, IR similarity scores (suspiciousness), and the nature of the affected code elements.
Key Insights and Results
Reducing Overfitting
Traditional APR tools often "overfit," creating patches that satisfy a single failing test but fail to address the underlying logic. iFixR's IR-based localization anchors the fix to the semantic description of the bug, which the authors found leads to more robust, less overfitted results.
Performance and Efficiency
- Repair Success: iFixR successfully generated genuine or plausible patches for 21 out of 44 Defects4J faults using its IR-based localizer.
- Ranking Accuracy: For the faults it could fix, iFixR placed a correct or plausible patch in the top-5 recommendations in 61% of cases (8 out of 13).
- Time-Saving: On average, iFixR could have localized bugs 26 days before the first reproducing test case was submitted to the project repositories, significantly accelerating the maintenance lifecycle.
Conclusion
iFixR demonstrates that bug reports are a viable and realistic driver for automated program repair. By shifting away from a dependency on failing test cases, iFixR aligns APR research with actual development practices, offering a tool that can suggest fixes nearly a month before traditional methods could even begin their work. Future work involves integrating iFixR with automated test generation to provide an even stronger oracle for correctness.
Summary updated