A Semi-Supervised and Inductive Embedding Model for Churn Prediction of Large-Scale Mobile Games.

Xi Liu, Muhe Xie, Xidao Wen, Rui Chen, Yong Ge, Nick Duffield, Na Wang

TL;DR

This paper introduces a novel semi-supervised, inductive graph embedding model to predict user churn in large-scale mobile games. By modeling user-game interactions as an attributed bipartite graph and jointly learning edge embeddings alongside churn probabilities, the approach successfully scales to hundreds of millions of interactions while seamlessly handling entirely new users and games.

What Problem Does This Paper Address?

Mobile gaming is a multi-billion dollar industry, and predicting when a user will quit a game (churn) is crucial for platforms to optimize marketing, retention campaigns, and app recommendations.

However, existing churn prediction models (like Random Forests or Logistic Regression) have mostly been tested on just a handful of games and fail to scale to real-world platforms hosting tens of thousands of apps. Furthermore, recent graph embedding techniques are typically transductive—meaning they learn fixed representations for specific nodes seen during training. In the real world, new users and new games appear daily. A practical system must be inductive, capable of generating predictions for unseen users and games on the fly. Finally, churn is an edge-level property (a specific user quitting a specific game), but most graph embedding methods are node-centric.

Methods

The authors propose a Deep Neural Network (DNN) architecture that operates on an attributed bipartite graph (where the two types of nodes are users and games). Key innovations include:

  • Inductive Edge Embedding: Instead of learning fixed embeddings for specific node IDs, the model learns an embedding function based on node attributes (e.g., device type, game genre). This allows it to generate embeddings for brand-new users or games simply by looking at their features.
  • Semi-Supervised Joint Training: The model is trained on two tasks simultaneously. The unsupervised component learns to infer graph context (which edges appear near each other), while the supervised component predicts the actual churn probability.
  • Attributed Random Walk: To train the unsupervised component, the authors need to sample "contextual" edges. Standard random walks only use graph topology (who is connected to whom). The authors introduce a walk that can jump to non-adjacent nodes if they share high attribute similarity, creating richer, more meaningful contexts.
  • Temporal Loss: The model explicitly accounts for graph dynamics. It encodes "temporal smoothness" (contexts evolve slowly) and "temporal dynamics" (the longer a user plays a game, the more likely they are to eventually churn). It also mathematically handles "right censoring"—the statistical challenge where recent data lacks definitive churn labels because the time window to define churn hasn't fully passed yet.

Datasets

The model was evaluated on two massive, real-world datasets collected independently over a 4-month period from the Samsung Game Launcher platform:

  • USA Dataset: 15,000 users, 19,705 games, and ~76.5 million play records.
  • Korea Dataset: 25,000 users, 18,470 games, and ~106.5 million play records.

Features included play history, game profiles (genre, developer, ratings), user info (device model, OS version), and rich contextual data like WiFi connection status, screen brightness, and audio volume.

Key Insights and Ideas

  • Edge-Centric over Node-Centric: Churn isn't just about a user's general behavior or a game's overall quality; it's about the specific relationship between the two. Directly embedding the edges of the bipartite graph captures this dynamic much better than combining separate user and game embeddings.
  • Similarity-Driven Contexts: In bipartite graphs, standard random walks bounce back and forth between two node types, often missing broader similarities. By allowing the random walk to traverse "invisible" augmented edges based on attribute similarity, the model learns that two users playing similar games are contextually related, even if they don't play the exact same game.
  • Time is a Feature: The authors observed a harsh reality of mobile gaming: 95% of user-game relationships end after 40 days. By forcing the model's loss function to respect this decaying temporal dynamic, the embeddings become much more aligned with the reality of player fatigue.

Key Results

The proposed Semi-Supervised (SS) model was compared against traditional ML baselines including Logistic Regression (LR), Decision Trees (DT), Random Forests (RF), and SVMs.

  • USA Dataset: The SS model achieved an AUC of 0.82 and a Recall of 0.78. For comparison, the strongest baseline (Random Forest) achieved an AUC of 0.75 and a Recall of just 0.31.
  • Korea Dataset: The SS model achieved an AUC of 0.82 and a Recall of 0.70 (vs. RF's 0.73 AUC and 0.27 Recall).

The massive boost in Recall (over 2.5x compared to Random Forests) is particularly valuable for business applications. It means the model successfully identifies a much larger proportion of actual churners, allowing platforms to target retention campaigns much more effectively without overfitting to a tiny subset of obvious churners.

Limitations and Future Work

  • Limitations: Due to privacy concerns, the authors did not have access to granular in-game data (e.g., player levels, specific in-game purchases, or achievements), which likely contain highly predictive signals for churn. Additionally, the model requires careful handling of right-censored data to avoid training on false negatives.
  • Future Work: The authors note that this inductive, edge-centric embedding framework is highly generalized. They suggest applying it to other bipartite churn problems outside of gaming, such as predicting subscription cancellations (e.g., Apple Music, Costco) or social network group unsubscriptions.

Summary updated

Abstract

Mobile gaming has emerged as a promising market with billion-dollar revenues. A variety of mobile game platforms and services have been developed around the world. One critical challenge for these platforms and services is to understand user churn behavior in mobile games. Accurate churn prediction will benefit many stakeholders such as game developers, advertisers, and platform operators. In this paper, we present the first large-scale churn prediction solution for mobile games. In view of the common limitations of the state-of-the-art methods built upon traditional machine learning models, we devise a novel semi-supervised and inductive embedding model that jointly learns the prediction function and the embedding function for user-app relationships. We model these two functions by deep neural networks with a unique edge embedding technique that is able to capture both contextual information and relationship dynamics. We also design a novel attributed random walk technique that takes into consideration both topological adjacency and attribute similarities. To evaluate the performance of our solution, we collect real-world data from the Samsung Game Launcher platform that includes tens of thousands of games and hundreds of millions of user-app interactions. The experimental results with this data demonstrate the superiority of our proposed model against existing state-of-the-art methods.
BibTeX
@inproceedings{liu2018semi,
  author = {Xi Liu and Muhe Xie and Xidao Wen and Rui Chen and Yong Ge and Nick Duffield and Na Wang},
  title = {{A Semi-Supervised and Inductive Embedding Model for Churn Prediction of Large-Scale Mobile Games.}},
  booktitle = {ICDM},
  year = {2018},
  url = {https://openreview.net/forum?id=OS0dWcn4-E},
}

Loading related content...

Figure