Prior-Based Dual Additive Latent Dirichlet Allocation for User-Item Connected Documents

Wei Zhang, Jianyong Wang

Abstract

This paper introduces Prior-based Dual Additive Latent Dirichlet Allocation (PDA-LDA), a generative topic model specifically engineered for documents that link users to items, such as product reviews or service tips. By incorporating user-specific and item-specific "topic factors" directly into an asymmetric Dirichlet prior at the document level, the model simultaneously captures individual user preferences and inherent item characteristics. Evaluated on large-scale datasets from Yelp and Amazon, PDA-LDA demonstrates superior performance over standard LDA and existing asymmetric variants in both text modeling (perplexity) and downstream classification tasks.

Introduction

Traditional topic models, most notably Latent Dirichlet Allocation (LDA), typically treat documents as independent entities within a corpus. However, in modern e-commerce and social media contexts, a document—such as a review—serves as a bridge between two distinct entities: a user and an item.

A review is rarely a neutral description; it is shaped by the user's personal biases (e.g., a preference for macro photography) and the item's actual attributes (e.g., a camera's zoom capability). Existing models often fail to capture this dual influence. For example, the Author-Topic Model (ATM) focuses on user preferences but neglects item attributes. PDA-LDA addresses this gap by creating a unified framework that automatically infers latent topic factors for both users and items from the text they co-generate.

Methodology: The PDA-LDA Model

The fundamental innovation of PDA-LDA lies in its treatment of the Dirichlet prior. While standard LDA uses a symmetric global prior ($\alpha$), PDA-LDA employs a document-level asymmetric prior tailored to the specific user-item pair associated with each document.

Additive Topic Factors

For a document $d$ written by user $u$ for item $v$, the model defines the Dirichlet prior $\alpha_d$ as an additive combination of three latent factors: $$\alpha_d^{u,v} = \exp(\phi_u + \phi_v + \phi_b)$$

  • $\phi_u$ (User Factor): Represents the user's persistent topical interests.
  • $\phi_v$ (Item Factor): Represents the topical characteristics or features of the item.
  • $\phi_b$ (Background Factor): Captures common language patterns and stop-word-like distributions that are neither user- nor item-specific.

The exponential transformation ensures that the resulting parameters remain positive, satisfying the requirements of the Dirichlet distribution.

Figure 1: Plate notation of the PDA-LDA model, illustrating how user (u) and item (v) factors influence the document-specific topic distribution (theta).

Inference and Learning

The authors utilize a Gibbs EM algorithm for parameter estimation. The process alternates between an E-step, which uses Collapsed Gibbs Sampling to assign latent topics to individual words, and an M-step, which employs gradient ascent to optimize the user and item topic factors ($\phi$) based on the current topic assignments.

Experimental Setup

The model was rigorously tested on three real-world datasets:

  • Yelp: 182,139 reviews across approximately 8,000 users and 5,000 items.
  • AmazonFood: 46,053 reviews.
  • AmazonSport: 7,982 reviews.

Data preprocessing included standard stop-word removal and filtering. Critically, the authors ensured each user and item had at least five associated documents to provide sufficient signal for the latent factors to be learned effectively.

Key Results and Discussion

Text Modeling Performance

The primary metric for evaluating topic models is perplexity, where a lower score indicates a better fit for the data. PDA-LDA consistently outperformed LDA, ATM, and Asymmetric-LDA (AS-LDA) across all test dimensions and datasets.

As shown in the results, moving from a corpus-level asymmetric prior (as in AS-LDA) to a document-level prior (PDA-LDA) yields significant gains. For instance, on the AmazonSport dataset, PDA-LDA achieved a perplexity of approximately 207, compared to LDA's 231 at the same topic dimension.

Classification and Profiling

Beyond perplexity, the authors evaluated the "discriminative power" of the learned topics by using them as features for a Support Vector Machine (SVM) to classify review sentiment. PDA-LDA's topics led to higher classification accuracy, suggesting that the additive factors help the model isolate content-rich topics from noise. Furthermore, qualitative analysis revealed that the learned factors $\phi_u$ and $\phi_v$ are sparse, accurately reflecting that users typically focus on a narrow set of interests and items are defined by a specific set of features.

Conclusion

PDA-LDA demonstrates that asymmetric priors are a powerful tool for incorporating metadata into generative models. By tailoring the prior to the user-item relationship, the model provides a more nuanced understanding of text data in collaborative environments. Future work may involve scaling the Gibbs EM approach for massive datasets and integrating explicit sentiment modeling directly into the additive factors to further enhance discriminative performance.

Summary updated

Abstract

User-item connected documents, such as customer reviews for specific items in online shopping website and user tips in location-based social networks, have become more and more prevalent recently. Inferring the topic distributions of user-item connected documents is beneficial for many applications, including document classification and summarization of users and items. While many different topic models have been proposed for modeling multiple text, most of them cannot account for the dual role of user-item connected documents (each document is related to one user and one item simultaneously) in topic distribution generation process. In this paper, we propose a novel probabilistic topic model called Prior-based Dual Additive Latent Dirichlet Allocation (PDA-LDA). It addresses the dual role of each document by associating its Dirichlet prior for topic distribution with user and item topic factors, which leads to a document-level asymmetric Dirichlet prior. In the experiments, we evaluate PDA-LDA on several real datasets and the results demonstrate that our model is effective in comparison to several other models, including held-out perplexity on modeling text and document classification application.
BibTeX
@inproceedings{zhang2015prior,
  author = {Wei Zhang and Jianyong Wang},
  title = {{Prior-Based Dual Additive Latent Dirichlet Allocation for User-Item Connected Documents}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2015},
}
Figure