Abstract
In large-scale recommender systems, balancing model complexity with real-time latency requirements is a persistent challenge. This paper details the development and deployment of the Decoupled Entity Representation Model (DERM) at Pinterest. DERM utilizes an upstream-downstream paradigm where sophisticated, high-capacity models are trained asynchronously to produce stable entity embeddings. These embeddings are then served to lightweight downstream ranking models in production. Our results demonstrate significant improvements in ad click-through rates (CTR) and conversion rates (CVR), alongside substantial infrastructure cost savings.
1. Introduction
Pinterest serves ads across diverse surfaces, including the Home Feed, Search, and Related Pins. Each surface requires real-time ranking within millisecond constraints. While advanced architectures like Transformers and Deep Hierarchical Ensemble Networks (DHEN) capture complex feature interactions, their computational overhead often precludes direct use in live ranking pipelines. DERM addresses this by decoupling the learning of rich entity representations from the real-time inference task.
2. The DERM Framework
The core of DERM is an upstream "teacher" model that learns representations offline, allowing for greater architectural complexity without impacting production latency.
2.1 Upstream Architecture
The upstream model employs a three-tower architecture: a User Tower, a Pin Tower, and an Interaction Tower. It utilizes the Deep Hierarchical Ensemble Network (DHEN) as a backbone, integrating modules such as MaskNet and Transformers to model intricate relationships between features.
2.2 Training and Stability
The model is trained using a multi-task supervised approach (predicting CTR and CVR) augmented by a self-supervised contrastive loss. This loss aligns user and item embeddings in a shared latent space. To maintain production stability, embeddings are refreshed daily using a weighted moving average, blending new updates with historical data to prevent volatility.
3. Experimental Results
The framework was evaluated using Pinterest's internal production datasets for ad engagement and conversions.
3.1 Performance Gains
Deployment of DERM led to a 1.38% platform-wide lift in CTR. Notably, the "Related Pins" surface saw a 2.83% improvement. For conversion-focused tasks, the model reduced Cost Per Action (CPA) by 1.61% and increased click conversion rates (cCVR) by 2.75%.
3.2 Operational Efficiency
By introducing a projection layer to optimize embedding dimensionality, the team increased offline training throughput by 5.68%, resulting in an estimated annual infrastructure saving of $212,000.
4. Key Insights
- Stability Trumps Recency: Experiments showed that a higher weight (0.8) on historical embeddings outperformed more frequent, volatile updates, suggesting that stable long-term representations are more effective for ranking.
- Cross-Domain Synergy: Embeddings trained for CVR tasks provided significant lifts when used in CTR models, indicating that different optimization goals capture complementary user-item signals.
5. Conclusion and Future Work
DERM successfully scales state-of-the-art deep learning for Pinterest Ads by decoupling representation learning from real-time ranking. Future efforts will focus on consolidating task-specific models into a unified entity space and transitioning toward content-based embeddings to better address cold-start scenarios and improve cross-domain transfer.
Summary updated