LLM-CompDroid: Repairing Configuration Compatibility Bugs in Android Apps with Pre-trained Large Language Models

Zhijie Liu, Yutian Tang, Meiyun Li, Xin Jin, Yunfei Long, Liang Feng Zhang, Xiapu Luo

TL;DR

LLM-CompDroid is a new hybrid framework that fixes "configuration compatibility bugs" in Android apps—those annoying glitches where an app looks or acts differently across various Android versions. By combining the precision of traditional dynamic analysis with the creative problem-solving of Large Language Models (LLMs), the tool achieves a 92.5% success rate, significantly outperforming both standalone AI and state-of-the-art rule-based tools.

What Problem Does This Paper Address?

Android developers face a "fragmentation" nightmare: an app must run smoothly on thousands of different device models and dozens of API levels. Much of an app's UI and behavior is defined in XML configuration files. However, these configurations often harbor compatibility bugs—for instance, an attribute that works perfectly on Android 12 might cause a crash or a visual glitch on Android 5.

Current solutions fall into two camps:

  1. Static Analysis (e.g., Lint): Fast but misses complex bugs because it doesn't "understand" the code's runtime context.
  2. Dynamic Analysis (e.g., ConfFix): More accurate but limited by rigid, pre-defined repair rules. If a bug requires a creative fix (like wrapping a UI element in a new container), these tools usually fail.

The authors set out to see if LLMs can bridge this gap, first by evaluating their raw performance and then by building a framework that plays to their strengths.

Methods

The researchers developed LLM-CompDroid, a framework that uses a "best of both worlds" approach. It operates in two main phases:

  1. The Systematic Study: They first benchmarked GPT-3.5, GPT-4, and Google Bard. They found that while LLMs are surprisingly bad at detecting bugs (high false-positive rates), they are remarkably flexible at repairing them—often suggesting fixes that traditional tools simply aren't programmed to consider.
  2. The Hybrid Framework: LLM-CompDroid first tries to fix "easy" bugs using ConfFix (a traditional dynamic analysis tool). If that fails, it hands the problem to an LLM-based multi-agent system. This system consists of three specialized agents:
  3. Repairer: Generates potential code patches.
  4. Checker: Acts as a critic, identifying compilation errors or logic flaws in the proposed patch.
  5. Optimizer: Refines the patch to ensure it is as concise as possible, removing unnecessary code.

The framework uses a Fitness Function to provide feedback. If a patch doesn't fix the visual discrepancy between API levels, the score tells the LLM to try again, creating an iterative loop of improvement.

Datasets

The study reused a well-established benchmark from previous research (Huang et al., 2023).

  • Scope: 13 popular open-source Android apps from GitHub.
  • Bugs: 77 confirmed configuration compatibility bugs.
  • API Range: Covers Android API levels 21 through 31. This dataset is particularly valuable because it includes "hard-to-repair" bugs that have historically defeated automated tools.

Key Insights and Ideas

  • LLMs are poor "detectives" but great "surgeons": The study revealed that LLMs lack the specific, up-to-date knowledge required to accurately identify API-level conflicts. However, once told exactly where the bug is, their ability to predict the correct XML structure is superior to rule-based systems.
  • The Power of Multi-Agent Collaboration: A single LLM prompt often produces "noisy" or slightly broken code. By splitting the task into a Repairer-Checker-Optimizer loop, the framework mimics a human code-review process, significantly boosting the quality of the final patch.
  • Overcoming Rule Rigidity: Traditional tools struggle with fixes that require structural changes (like adding a FrameLayout). LLMs can "hallucinate" these structures correctly because they have seen millions of similar UI patterns in their training data.

Key Results

The results show a clear hierarchy of performance:

  • LLM-CompDroid (GPT-4 version): Achieved a 92.5% Correct repair rate.
  • ConfFix (State-of-the-art traditional tool): Achieved 82.1%.
  • Pure GPT-4: Only managed 53.8% on its own.
  • Google Bard: Performed the worst, with only a 2.5% success rate, largely due to "hallucinating" non-existent XML attributes and resources.

Crucially, LLM-CompDroid successfully repaired bugs that were previously considered "unfixable" by automated means, proving that AI can handle the "long tail" of complex software engineering tasks.

Limitations and Future Work

Despite the high success rate, the authors noted a few hurdles:

  • The "android:gravity" Problem: LLMs frequently struggled with (or rejected) fixes involving the android:gravity attribute, likely because the visual impact of these bugs is often subtle, leading to fewer "correct" examples in the LLM's training data.
  • Manual Assessment: Currently, verifying if a UI "looks right" still requires some manual oversight by humans to avoid subjectivity.
  • Dataset Size: While 77 bugs is standard for this niche, expanding to a larger, more diverse dataset would further validate the framework's robustness.

Future work will likely focus on making the "Checker" agent even smarter—perhaps by integrating real-time UI rendering to give the LLM visual feedback on its repairs.

Summary updated

Abstract

XML configurations are integral to the Android development framework, particularly in the realm of UI display. However, these configurations can introduce compatibility issues (bugs), resulting in divergent visual outcomes and system crashes across various Android API versions (levels). In this study, we systematically investigate LLM-based approaches for detecting and repairing configuration compatibility bugs. Our findings highlight certain limitations of LLMs in effectively identifying and resolving these bugs, while also revealing their potential in addressing complex, hard-to-repair issues that traditional tools struggle with. Leveraging these insights, we introduce the LLM-CompDroid framework, which combines the strengths of LLMs and traditional tools for bug resolution. Our experimental results demonstrate a significant enhancement in bug resolution performance by LLM-CompDroid, with LLM-CompDroid-GPT-3.5 and LLM-CompDroid-GPT-4 surpassing the state-of-the-art tool, ConfFix, by at least 9.8% and 10.4% in both Correct and Correct@k metrics, respectively. This innovative approach holds promise for advancing the reliability and robustness of Android applications, making a valuable contribution to the field of software development.
BibTeX
@article{liu2025llm,
  author = {Zhijie Liu and Yutian Tang and Meiyun Li and Xin Jin and Yunfei Long and Liang Feng Zhang and Xiapu Luo},
  title = {{LLM-CompDroid: Repairing Configuration Compatibility Bugs in Android Apps with Pre-trained Large Language Models}},
  journal = {ACM Transactions on Software Engineering and Methodology},
  year = {2025},
  url = {https://openreview.net/forum?id=UlTbthHfAy},
}

Loading related content...

Figure