Abstract
This paper introduces Prior-based Dual Additive Latent Dirichlet Allocation (PDA-LDA), a generative topic model specifically engineered for documents that link users to items, such as product reviews or service tips. By incorporating user-specific and item-specific "topic factors" directly into an asymmetric Dirichlet prior at the document level, the model simultaneously captures individual user preferences and inherent item characteristics. Evaluated on large-scale datasets from Yelp and Amazon, PDA-LDA demonstrates superior performance over standard LDA and existing asymmetric variants in both text modeling (perplexity) and downstream classification tasks.
Introduction
Traditional topic models, most notably Latent Dirichlet Allocation (LDA), typically treat documents as independent entities within a corpus. However, in modern e-commerce and social media contexts, a document—such as a review—serves as a bridge between two distinct entities: a user and an item.
A review is rarely a neutral description; it is shaped by the user's personal biases (e.g., a preference for macro photography) and the item's actual attributes (e.g., a camera's zoom capability). Existing models often fail to capture this dual influence. For example, the Author-Topic Model (ATM) focuses on user preferences but neglects item attributes. PDA-LDA addresses this gap by creating a unified framework that automatically infers latent topic factors for both users and items from the text they co-generate.
Methodology: The PDA-LDA Model
The fundamental innovation of PDA-LDA lies in its treatment of the Dirichlet prior. While standard LDA uses a symmetric global prior ($\alpha$), PDA-LDA employs a document-level asymmetric prior tailored to the specific user-item pair associated with each document.
Additive Topic Factors
For a document $d$ written by user $u$ for item $v$, the model defines the Dirichlet prior $\alpha_d$ as an additive combination of three latent factors: $$\alpha_d^{u,v} = \exp(\phi_u + \phi_v + \phi_b)$$
- $\phi_u$ (User Factor): Represents the user's persistent topical interests.
- $\phi_v$ (Item Factor): Represents the topical characteristics or features of the item.
- $\phi_b$ (Background Factor): Captures common language patterns and stop-word-like distributions that are neither user- nor item-specific.
The exponential transformation ensures that the resulting parameters remain positive, satisfying the requirements of the Dirichlet distribution.
Inference and Learning
The authors utilize a Gibbs EM algorithm for parameter estimation. The process alternates between an E-step, which uses Collapsed Gibbs Sampling to assign latent topics to individual words, and an M-step, which employs gradient ascent to optimize the user and item topic factors ($\phi$) based on the current topic assignments.
Experimental Setup
The model was rigorously tested on three real-world datasets:
- Yelp: 182,139 reviews across approximately 8,000 users and 5,000 items.
- AmazonFood: 46,053 reviews.
- AmazonSport: 7,982 reviews.
Data preprocessing included standard stop-word removal and filtering. Critically, the authors ensured each user and item had at least five associated documents to provide sufficient signal for the latent factors to be learned effectively.
Key Results and Discussion
Text Modeling Performance
The primary metric for evaluating topic models is perplexity, where a lower score indicates a better fit for the data. PDA-LDA consistently outperformed LDA, ATM, and Asymmetric-LDA (AS-LDA) across all test dimensions and datasets.
As shown in the results, moving from a corpus-level asymmetric prior (as in AS-LDA) to a document-level prior (PDA-LDA) yields significant gains. For instance, on the AmazonSport dataset, PDA-LDA achieved a perplexity of approximately 207, compared to LDA's 231 at the same topic dimension.
Classification and Profiling
Beyond perplexity, the authors evaluated the "discriminative power" of the learned topics by using them as features for a Support Vector Machine (SVM) to classify review sentiment. PDA-LDA's topics led to higher classification accuracy, suggesting that the additive factors help the model isolate content-rich topics from noise. Furthermore, qualitative analysis revealed that the learned factors $\phi_u$ and $\phi_v$ are sparse, accurately reflecting that users typically focus on a narrow set of interests and items are defined by a specific set of features.
Conclusion
PDA-LDA demonstrates that asymmetric priors are a powerful tool for incorporating metadata into generative models. By tailoring the prior to the user-item relationship, the model provides a more nuanced understanding of text data in collaborative environments. Future work may involve scaling the Gibbs EM approach for massive datasets and integrating explicit sentiment modeling directly into the additive factors to further enhance discriminative performance.
Summary updated