A Multi-Model Approach for User Portrait

Yanbo Chen, Jingsha He, Wei Wei, Nafei Zhu, Cong YuFuture 2021 · DOI

TL;DR

This paper introduces a novel two-stage, multi-model fusion approach for creating user portraits by predicting demographic attributes like age, gender, and education. By combining the strengths of various machine learning and deep learning models in a staged ensemble, the authors achieve higher accuracy and better generalization than single-model methods, paving the way for more personalized online experiences.

What Problem Does This Paper Address?

In today's digital landscape, "information overload" is a significant challenge. To combat this, personalized services and recommendations are crucial, and at their core lies the concept of a "user portrait." A user portrait is essentially a collection of demographic and behavioral attributes that helps service providers understand their users better, enabling them to offer tailored services and enhance user experience. However, accurately predicting these attributes, especially from unstructured text data like search queries, is a complex task. Existing methods often struggle with the nuances of natural language and the inherent sparsity of user data. This paper aims to address this by proposing a more robust and accurate method for constructing user portraits through a sophisticated multi-model approach.

Methods

The proposed solution is a two-stage, multi-model fusion architecture.

Stage 1: Base Models This stage employs three parallel sub-tasks, each using different models to predict user attributes:

  • TF-IDF + SVM: Leverages traditional machine learning (Support Vector Machine) with TF-IDF for feature extraction, focusing on word frequency and inverse document frequency to capture user habits.
  • Doc2vec + NN: Utilizes Doc2vec to generate vector representations of documents (search queries) and then feeds these into a neural network for classification. This captures semantic relationships within the text.
  • Word2vec + CNN + LSTM: This is a deep learning pipeline. Word2vec creates word embeddings, CNNs extract local features (like n-grams), and LSTMs capture sequential dependencies and contextual information within the short text.

The outputs from these three sub-tasks (which predict gender, age, and education) are then combined.

Stage 2: Fusion Model The outputs from Stage 1 are fed into a second-stage model. Here, the authors use XGBoost, a powerful gradient boosting algorithm, combined with a stacking ensemble technique. Stacking involves training a meta-model (XGBoost in this case) on the predictions of the base models. This allows the system to learn how to best combine the diverse predictions from the initial models, further refining the accuracy and generalization.

Datasets

The experiment utilized a dataset derived from the "Search Dog User Portrait Mining in Big Data Precision Marketing" initiative by the Chinese Computer Federation (CCF) in 2016. This dataset contains one month of user search queries and corresponding demographic labels (gender, age, and education). The training dataset comprised 100,000 data items, and the test dataset also contained 100,000 items.

Key Insights and Ideas

  • Ensemble Power: The core idea is that combining multiple diverse models (traditional ML, Doc2vec, and CNN+LSTM) in a staged approach yields better results than any single model. Each model captures different aspects of the user data.
  • Two-Stage Advantage: The staged approach, where base model predictions are used as input for a meta-model (XGBoost), allows for a more sophisticated learning of how to integrate information, leading to improved accuracy and generalization.
  • Handling Short Text: The paper highlights the challenge of short, unstructured text (search queries) and demonstrates how a combination of CNN and LSTM can effectively extract features and context from such data.
  • XGBoost as a Meta-Learner: Using XGBoost in the second stage proves effective for learning from the diverse outputs of the first-stage models, effectively "fusing" their predictions.
  • Addressing Data Imbalance: The authors note that their chosen evaluation metric (accuracy) makes downsampling and upsampling unnecessary for imbalanced datasets, and XGBoost can effectively learn optimal thresholds.

Key Results

The proposed multi-stage, multi-model fusion approach achieved the highest prediction accuracy across all demographic attributes (Education, Age, and Gender) compared to individual models like TF-IDF+SVM, DBOW+NN, DM+NN, and Word2vec+CNN+LSTM. Specifically:

  • Education: The multi-stage model achieved an accuracy of 0.5946.
  • Age: The multi-stage model achieved an accuracy of 0.6153.
  • Gender: The multi-stage model achieved an accuracy of 0.8255.

These results represent a significant improvement over the single models, demonstrating the effectiveness of the fusion strategy. The paper also shows a variation diagram of iteration times and accuracy, indicating that the models converge and achieve stable performance.

Limitations and Future Work

The authors acknowledge that while their proposed model improves accuracy, there is still room for further enhancement. They suggest that future work could involve:

  • More Experiments: Conducting more experiments with different fusion models and techniques.
  • Applying Other Techniques: Exploring other advanced techniques to further boost prediction accuracy.
  • Exploring Other Models: While not explicitly stated as a limitation, the paper implies that other models might exist that could yield even better results, and their fusion could be explored.

Summary updated

Abstract

Age, gender, educational background, and so on are the most basic attributes for identifying and portraying users. It is also possible to conduct in-depth mining analysis and high-level predictions based on such attributes to learn users’ preferences and personalities so as to enhance users’ online experience and to realize personalized services in real applications. In this paper, we propose using classification algorithms in machine learning to predict users’ demographic attributes, such as gender, age, and educational background, based on one month of data collected with the Sogou search engine with the goal of making user portraits. A multi-model approach using the fusion algorithms is adopted and hereby described in the paper. The proposed model is a two-stage structure using one month of data with demographic labels as the training data. The first stage of the structure is based on traditional machine learning models and neural network models, whereas the second one is a combination of the models from the first stage. Experimental results show that our proposed multi-model method can achieve more accurate results than the single-model methods in predicting user attributes. The proposed approach also has stronger generalization ability in predicting users’ demographic attributes, making it more adequate to profile users.
BibTeX
@inproceedings{chen2021multi,
  author = {Yanbo Chen and Jingsha He and Wei Wei and Nafei Zhu and Cong Yu},
  title = {{A Multi-Model Approach for User Portrait}},
  booktitle = {Future},
  year = {2021},
  doi = {10.3390/fi13060147},
}
Figure