Introduction
As social media consumption grows exponentially, the challenge of effectively modeling user interests against heterogeneous social content becomes increasingly critical. Traditional recommendation systems often struggle with "content-centric" networks like Twitter or Pinterest due to two primary hurdles: the Heterogeneity Gap (the mix of text, images, hashtags, and URLs) and the Semantic Gap (the disconnect between raw data and underlying user intent).
To bridge these gaps, we present D-Sempre (Deep Semantic-Preserving Embedding), a framework designed to map users and multi-modal content into a shared latent space by capturing hidden semantic correlations.
The D-Sempre Framework
The core of D-Sempre is a two-branch neural network architecture that processes social posts and user profiles separately before projecting them into a unified embedding space.
1. Social Post Branch
This branch handles the multi-modal nature of social media. It utilizes a 5-layer ConvNet for visual features and a Word2Vec-based encoder for textual data. A key innovation in D-Sempre is data expansion: the model doesn't just analyze the tweet itself; it crawls linked URLs and uses Google Reverse Image Search to retrieve the original context of images, providing a much cleaner semantic signal than a character-limited post.
2. User Branch
Rather than relying solely on a User ID, D-Sempre builds a rich descriptor based on the user’s profile and their social relations (the profiles of people they follow). These are processed via Word2Vec and k-means clustering into semantic "bags-of-words" that represent the user's broader interest sphere.
Dual-Constraint Loss
The model is trained using a stochastic hinge loss with a unique dual-constraint approach:
- Cross-Instance Distance Constraint: Ensures a user's embedding is closer to posts they interacted with than to random, irrelevant posts.
- Within-Instance Semantic-Preserving Constraint: This novel constraint forces posts belonging to the same semantic cluster (e.g., "Artificial Intelligence") to remain close in the latent space, regardless of which user posted them. This acts as a powerful regularizer, helping the model generalize to new content and mitigating the "cold start" problem.
Experimental Results
We evaluated D-Sempre on a custom Twitter dataset comprising approximately 3.7 million tweets from 586 active users. The framework significantly outperformed traditional collaborative filtering and existing deep learning baselines:
- Precision: D-Sempre achieved a P@1 of 0.70, a substantial improvement over the best-performing baseline (CDL at ~0.48).
- Ablation Insights: The inclusion of the semantic-preserving constraint consistently improved performance, proving that maintaining the internal structure of the content space is vital for accurate ranking.
- Contextual Value: Adding external context (URLs and Google Search data) provided a larger performance boost than internal metadata like hashtags, suggesting that "hidden" extrinsic information is the most valuable for understanding social content.
Conclusion
D-Sempre demonstrates that by integrating multi-modal data and expanding social posts with extrinsic context, we can effectively bridge the semantic gap in social media modeling. While the dependency on external APIs presents a potential production bottleneck, the framework's ability to preserve semantic structure offers a robust path forward for personalized recommendation and social media summarization.
Summary updated