LLM-CompDroid: Repairing Configuration Compatibility Bugs in Android Apps with Pre-trained Large Language Models

Zhijie Liu, Yutian Tang, Meiyun Li, Xin Jin, Yunfei Long, Liang Feng Zhang, Xiapu Luo

Abstract

XML configurations are central to Android UI development but often introduce compatibility bugs across different API levels. This study investigates the use of Large Language Models (LLMs) for detecting and repairing these bugs. We introduce LLM-CompDroid, a hybrid framework that integrates LLMs with traditional dynamic analysis. Our results show that LLM-CompDroid significantly outperforms existing tools, achieving a 92.5% success rate and resolving complex issues that traditional rule-based systems cannot handle.

Introduction: The Fragmentation Challenge

Android developers must navigate a highly fragmented ecosystem, ensuring applications function correctly across thousands of device models and numerous API levels. Much of an app's visual layout and behavior is defined in XML configuration files. However, these configurations frequently harbor compatibility bugs—instances where a specific attribute or structure behaves differently across Android versions, leading to visual discrepancies or system crashes.

Visual discrepancy of a compatibility bug across different Android API levels.

Traditional solutions like static analysis (e.g., Android Lint) often miss runtime context, while dynamic analysis tools like ConfFix are limited by rigid, pre-defined repair rules. This paper explores whether LLMs can provide the flexibility needed to address these "long-tail" compatibility issues.

Methodology: The LLM-CompDroid Framework

The researchers developed LLM-CompDroid, a hybrid framework designed to leverage the complementary strengths of traditional tools and LLMs. The approach consists of two primary stages:

1. Systematic Study of LLM Capabilities

Before building the framework, the team benchmarked GPT-3.5, GPT-4, and Google Bard on their ability to identify and fix configuration bugs. The study revealed a critical insight: while LLMs are relatively poor at detecting bugs (often yielding high false-positive rates), they excel at repairing them once the location is identified.

Benchmarking results showing LLM performance in recalling and precisely identifying XML elements and attributes.

2. The Hybrid Repair Pipeline

LLM-CompDroid employs a tiered strategy:

  • Initial Repair: It first attempts to resolve "simple" bugs using ConfFix, a state-of-the-art dynamic analysis tool.
  • LLM Multi-Agent System: If traditional methods fail, the task is handed to a specialized multi-agent system consisting of a Repairer (generates patches), a Checker (evaluates logic), and an Optimizer (refines code).
  • Feedback Loop: A fitness function evaluates the visual output of the patched app across API levels, providing iterative feedback to the LLM until the discrepancy is resolved.

Evaluation and Key Results

The framework was tested on a benchmark of 77 confirmed configuration compatibility bugs across 13 popular open-source Android apps, covering API levels 21 through 31.

The results demonstrated a clear advantage for the hybrid approach:

  • LLM-CompDroid (GPT-4): Achieved a 92.5% correct repair rate.
  • ConfFix: Achieved 82.1%.
  • Standalone GPT-4: Only managed 53.8%.
  • Google Bard: Performed poorly at 2.5%, frequently hallucinating non-existent XML attributes.

Key Insights

  • Surgeons, not Detectives: LLMs lack the precise, version-specific knowledge to reliably detect API conflicts but are highly effective at predicting correct XML structures when guided.
  • Structural Flexibility: Unlike rule-based tools, LLMs can suggest complex structural changes, such as wrapping elements in new containers (e.g., FrameLayout), which are often required for sophisticated UI fixes.
  • Multi-Agent Synergy: The Repairer-Checker-Optimizer loop significantly reduces the "noise" and errors typical of single-prompt LLM outputs.

Limitations and Future Outlook

While highly effective, LLM-CompDroid faces challenges with subtle visual bugs, such as those involving android:gravity, which may be underrepresented in training data. Additionally, verifying UI "correctness" still involves a degree of manual oversight. Future work aims to integrate real-time UI rendering into the "Checker" agent to provide the LLM with direct visual feedback, further automating the verification process.

Conclusion

LLM-CompDroid represents a significant step forward in automated software repair. By combining the reliability of dynamic analysis with the creative problem-solving capabilities of LLMs, the framework provides a robust solution for the complex, fragmented world of Android development.

Summary updated

Abstract

XML configurations are integral to the Android development framework, particularly in the realm of UI display. However, these configurations can introduce compatibility issues (bugs), resulting in divergent visual outcomes and system crashes across various Android API versions (levels). In this study, we systematically investigate LLM-based approaches for detecting and repairing configuration compatibility bugs. Our findings highlight certain limitations of LLMs in effectively identifying and resolving these bugs, while also revealing their potential in addressing complex, hard-to-repair issues that traditional tools struggle with. Leveraging these insights, we introduce the LLM-CompDroid framework, which combines the strengths of LLMs and traditional tools for bug resolution. Our experimental results demonstrate a significant enhancement in bug resolution performance by LLM-CompDroid, with LLM-CompDroid-GPT-3.5 and LLM-CompDroid-GPT-4 surpassing the state-of-the-art tool, ConfFix, by at least 9.8% and 10.4% in both Correct and Correct@k metrics, respectively. This innovative approach holds promise for advancing the reliability and robustness of Android applications, making a valuable contribution to the field of software development.
BibTeX
@article{liu2025llm,
  author = {Zhijie Liu and Yutian Tang and Meiyun Li and Xin Jin and Yunfei Long and Liang Feng Zhang and Xiapu Luo},
  title = {{LLM-CompDroid: Repairing Configuration Compatibility Bugs in Android Apps with Pre-trained Large Language Models}},
  journal = {ACM Transactions on Software Engineering and Methodology},
  year = {2025},
  url = {https://openreview.net/forum?id=UlTbthHfAy},
}
Figure