Detecting Low Rating Android Apps Before They Have Reached the Market

Ding Li, Dongjin SongarXiv 2017 · arXiv

Abstract

In the crowded Android ecosystem, low-rated apps damage the reputation of app stores and drive users away. Current quality control is reactive, relying on user reviews after publication. This paper presents Sextant, a novel system that predicts whether an Android app will receive a low user rating using only its raw .apk file. By combining static program analysis with deep learning, Sextant achieves a 92.31% accuracy, offering a proactive way for app markets to protect their reputation and for developers to fix "bad" apps early.

Introduction

The Android ecosystem faces a significant challenge: "bad apples"—apps with poor UI, buggy code, or inferior user experiences—that negatively impact user trust and store reputation. Traditional quality control is reactive; an app is published, users suffer, leave one-star reviews, and only then is the app penalized. Manual inspection is unfeasible given the volume of daily uploads. Furthermore, "app quality" is subjective, making it difficult for traditional tools focused on security or crashes to model the general user experience.

Methodology: The Sextant System

Sextant introduces a three-stage pipeline to treat app rating prediction as a binary classification problem.

The Sextant system architecture, illustrating the three-stage pipeline from APK analysis to final classification.

1. Analysis Stage (Feature Extraction)

Sextant extracts two novel representations from the APK:

  • Semantic Vector: Captures the logic of executable (.dex) files. Beyond instruction frequency, it tracks loop depth and branch counts to understand structural complexity.
  • Layout Vector: Parses UI XML files to model visual structure, tracking UI element types and their nesting depth in the layout tree.

2. Pre-Training Stage

Because code and UI data differ in scale and structure, Sextant employs two separate Convolutional Neural Networks (CNNs). These networks learn unified, high-level features from the raw structural data.

3. Learning Stage

The features from both CNNs are concatenated and processed by a Multi-Layer Perceptron (MLP), which performs the final classification: "low rating" or "not low rating."

Experimental Evaluation

The system was evaluated using a dataset of 33,456 Android apps crawled from Google Play. The dataset was diverse, spanning 25 categories (with Games comprising 28%). Apps were labeled "low rating" if they had fewer than three stars.

Key Results

Sextant achieved an average accuracy of 92.31%, with a precision of 90.50% and a recall of 94.31%.

Performance comparison of different feature sets, showing that the combined 'Union' model achieves the highest accuracy and recall.

As shown in the performance comparison, the "Union" model (Sextant) significantly outperforms traditional "Bag of Words" (BOW) approaches and models relying solely on code (Executable) or UI features in isolation.

Discussion and Insights

  • Structural Context: Adding structural context (loop depth, branching) improved accuracy by over 3.6% compared to simple instruction frequency. This suggests user frustration correlates with complex, unoptimized logic.
  • Multimodal Synergy: The integration of UI layout and code semantics provides a more holistic view of app quality than either feature set alone.
  • Efficiency: The static analysis phase averages 23.98 seconds per app, making it viable for real-time app store submission pipelines. The model can be trained on over 30,000 apps in approximately 3.4 hours on consumer-grade hardware.

Limitations and Future Work

The study identifies a "borderline" problem where apps near the 3-star cutoff are harder to classify. Additionally, the dataset uses apps from 2014; while core structures remain similar, modern Android frameworks may require updated feature extraction methods. Future work could also address processing bottlenecks in call graph generation to further improve speed.

Summary updated

BibTeX
@misc{li2017detecting,
  author = {Ding Li and Dongjin Song},
  title = {{Detecting Low Rating Android Apps Before They Have Reached the Market}},
  year = {2017},
  eprint = {1712.05843},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1712.05843},
}
Figure