Cognitive Inference of Demographic Data by User Ratings

Shangguang WangarXiv 2017 · arXiv

TL;DR

This paper introduces R2P, a novel model that infers user demographics (like gender and age) from their online ratings. By jointly learning to extract features from ratings and classify demographics, R2P significantly improves inference accuracy compared to existing methods, demonstrating the untapped potential of user feedback data.

What Problem Does This Paper Address?

In today's digital landscape, understanding user demographics is crucial for personalized recommendations and targeted marketing. However, this data is often private and unavailable. While previous research has explored inferring demographics from various online activities like browsing history or social media interactions, a significant gap exists in leveraging user-generated ratings of products and services. Users readily provide ratings for movies, apps, and restaurants, but this rich source of information has largely been underutilized for demographic inference. The challenge lies in effectively extracting meaningful patterns from sparse and potentially noisy rating data and simultaneously inferring multiple demographic attributes while accounting for their interdependencies.

Methods

The core of this paper is the Rating to Profile (R2P) model. R2P tackles the problem by framing demographic inference as a multi-task classification problem, aiming to predict multiple demographic attributes (gender and age) simultaneously.

Here's a breakdown of its key methodological contributions:

  • Joint Demographic Representation: R2P first constructs a "demographic joint representation" by concatenating one-hot encoded labels for each demographic attribute (e.g., male/female for gender, young/adult/middle-age/old for age). This creates a unified representation that captures the relationships between different demographic attributes.
  • Integrated Feature Extraction and Classification: Unlike many prior methods that train these components separately, R2P integrates them. It uses a "representation learning" approach to automatically learn informative "representation vectors" for each item (e.g., movie, app) from the user ratings data. This learned representation then serves as input to a classifier that infers the demographic joint representation. This joint training allows the model to optimize both feature extraction and classification for better overall performance.
  • Leveraging Rating Values: R2P goes beyond simply using the presence or absence of a rating. It incorporates the actual rating value (e.g., 1-5 stars) as a scaling factor in its probability model, assuming higher ratings indicate a stronger preference. This allows the model to better capture user preferences.
  • Negative Sampling for Training: To efficiently train the model on large datasets, R2P employs a negative sampling technique. This involves training the model to distinguish between the true demographic representation and several randomly sampled "negative" representations, thereby improving the accuracy of the learned model.

Datasets

The authors constructed two novel datasets for their experiments:

  • MovieRatings: This dataset was built by crawling Douban, a Chinese movie review platform. It contains approximately 243,000 ratings from 1,452 users for 2,300 movies, with self-reported gender and age information. Ratings were normalized to a [0, 1] range.
  • MobilePackageRatings: This dataset was collected from a mobile operator in Shanghai, China. It includes data traffic records for 4,418 users, along with their gender, age, and the mobile data bundles they used. User ratings were derived from the total time spent on each data bundle, normalized using Min-Max scaling.

The paper notes that there are no readily available benchmark datasets for user demographic inference from ratings, necessitating the creation of these new datasets.

Key Insights and Ideas

  • User ratings are a rich, untapped source for demographic inference: The paper strongly suggests that the implicit information within user ratings can significantly boost demographic inference accuracy, a direction previously underexplored.
  • Jointly learning features and demographics is superior: By integrating the feature extraction (item representation learning) and classification (demographic inference) processes, R2P achieves better performance than methods that train these components independently. This highlights the benefit of a holistic approach.
  • Interdependencies between demographic attributes matter: The "demographic joint representation" effectively captures how different demographic attributes relate to each other, leading to more accurate inferences. For instance, the model learns that certain gender-age combinations are more common or have distinct rating patterns.
  • User activeness is a key factor: More active users (those who provide more ratings) lead to significantly better demographic inference performance. This suggests that encouraging user engagement with rating systems could indirectly improve demographic understanding.

Key Results

R2P demonstrated superior performance across both the MovieRatings and MobilePackageRatings datasets compared to several state-of-the-art methods, including SVD-Single, SVD-Multiple, JNE, and SNE.

  • Significant Improvement: On the MovieRatings dataset, R2P achieved a relative improvement of 21.2% in micro-F1 score over the best-performing prior method (SNE).
  • Consistent Performance: Similar trends were observed on the MobilePackageRatings dataset, although the overall performance was slightly lower, likely due to dataset characteristics.
  • Parameter Sensitivity: Experiments showed that R2P's performance is sensitive to training ratio and the dimension of representation vectors. Optimal performance was achieved with a training ratio of 90% and specific dimension values (D=70 for MovieRatings, D=14 for MobilePackageRatings). The regularization constant and negative sampling number were also analyzed, with specific values identified for efficient and effective training.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Sparsity and Cold-Start Problem: R2P performs poorly when data is extremely sparse or for "cold-start" users (those who haven't rated anything). Future work could explore incorporating other side information, such as user comments or social network data, to address this.
  • Domain Specificity: While R2P was tested on movie and mobile data bundles, its performance on other domains is yet to be fully explored. The authors plan to construct more datasets to validate its generalizability.
  • Leveraging Textual Data: The paper suggests extending R2P to incorporate textual data from user comments, hypothesizing that combining content analysis with ratings could further enhance demographic inference.
  • Exploiting Inferred Demographics: The current work focuses solely on inference. Future research could explore how these inferred demographics can be effectively used for downstream tasks like improving user modeling and recommendation systems, particularly for addressing the cold-start problem.

Summary updated

BibTeX
@misc{wang2017cognitive,
  author = {Shangguang Wang},
  title = {{Cognitive Inference of Demographic Data by User Ratings}},
  year = {2017},
  eprint = {1703.04216},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1703.04216},
}
Figure