D-Sempre: Learning Deep Semantic-Preserving Embeddings for User interests-Social Contents Modeling

Shuang Ma, Chang Wen ChenarXiv 2018 · OpenReview

Introduction

As social media consumption grows exponentially, the challenge of effectively modeling user interests against heterogeneous social content becomes increasingly critical. Traditional recommendation systems often struggle with "content-centric" networks like Twitter or Pinterest due to two primary hurdles: the Heterogeneity Gap (the mix of text, images, hashtags, and URLs) and the Semantic Gap (the disconnect between raw data and underlying user intent).

To bridge these gaps, we present D-Sempre (Deep Semantic-Preserving Embedding), a framework designed to map users and multi-modal content into a shared latent space by capturing hidden semantic correlations.

Conceptual illustration of bridging the semantic gap (a) and mapping users to interest clusters (b) in a shared latent space.

The D-Sempre Framework

The core of D-Sempre is a two-branch neural network architecture that processes social posts and user profiles separately before projecting them into a unified embedding space.

1. Social Post Branch

This branch handles the multi-modal nature of social media. It utilizes a 5-layer ConvNet for visual features and a Word2Vec-based encoder for textual data. A key innovation in D-Sempre is data expansion: the model doesn't just analyze the tweet itself; it crawls linked URLs and uses Google Reverse Image Search to retrieve the original context of images, providing a much cleaner semantic signal than a character-limited post.

2. User Branch

Rather than relying solely on a User ID, D-Sempre builds a rich descriptor based on the user’s profile and their social relations (the profiles of people they follow). These are processed via Word2Vec and k-means clustering into semantic "bags-of-words" that represent the user's broader interest sphere.

Overview of the data mining process, showing (a) extrinsic context expansion via URLs and image search, and (b) user modeling via profiles and social relations.

Dual-Constraint Loss

The model is trained using a stochastic hinge loss with a unique dual-constraint approach:

  • Cross-Instance Distance Constraint: Ensures a user's embedding is closer to posts they interacted with than to random, irrelevant posts.
  • Within-Instance Semantic-Preserving Constraint: This novel constraint forces posts belonging to the same semantic cluster (e.g., "Artificial Intelligence") to remain close in the latent space, regardless of which user posted them. This acts as a powerful regularizer, helping the model generalize to new content and mitigating the "cold start" problem.

Experimental Results

We evaluated D-Sempre on a custom Twitter dataset comprising approximately 3.7 million tweets from 586 active users. The framework significantly outperformed traditional collaborative filtering and existing deep learning baselines:

  • Precision: D-Sempre achieved a P@1 of 0.70, a substantial improvement over the best-performing baseline (CDL at ~0.48).
  • Ablation Insights: The inclusion of the semantic-preserving constraint consistently improved performance, proving that maintaining the internal structure of the content space is vital for accurate ranking.
  • Contextual Value: Adding external context (URLs and Google Search data) provided a larger performance boost than internal metadata like hashtags, suggesting that "hidden" extrinsic information is the most valuable for understanding social content.

Conclusion

D-Sempre demonstrates that by integrating multi-modal data and expanding social posts with extrinsic context, we can effectively bridge the semantic gap in social media modeling. While the dependency on external APIs presents a potential production bottleneck, the framework's ability to preserve semantic structure offers a robust path forward for personalized recommendation and social media summarization.

Summary updated

Abstract

Exponential growth of social media consumption demands effective user interests-social contents modeling for more personalized recommendation and social media summarization. However, due to the heterogeneous nature of social contents, traditional approaches lack the ability of capturing the hidden semantic correlations across these multi-modal data, which leads to semantic gaps between social content understanding and user interests. To effectively bridge the semantic gaps, we propose a novel deep learning framework for user interests-social contents modeling. We first mine and parse data, i.e. textual content, visual content, social context and social relation, from heterogeneous social media feeds. Then, we design a two-branch network to map the social contents and users into a same latent space. Particularly, the network is trained by a large margin objective that combines a cross-instance distance constraint with a within-instance semantic-preserving constraint in an end-to- end manner. At last, a Deep Semantic-Preserving Embedding (D-Sempre) is learned, and the ranking results can be given by calculating distances between social contents and users. To demonstrate the effectiveness of D-Sempre in user interests-social contents modeling, we construct a Twitter dataset and conduct extensive experiments on it. As a result, D-Sempre effectively integrates the multi-modal data from heterogeneous social media feeds and captures the hidden semantic correlations between users' interests and social contents.
BibTeX
@misc{ma2018d,
  author = {Shuang Ma and Chang Wen Chen},
  title = {{D-Sempre: Learning Deep Semantic-Preserving Embeddings for User interests-Social Contents Modeling}},
  year = {2018},
  eprint = {1802.06451},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1802.06451},
}
Figure