Introduction
In recommender systems, content-based recommendation (CBR) is vital when user feedback is sparse. However, effectively extracting features from text remains a challenge: simple models often miss linguistic nuances, while complex neural networks are prone to overfitting. Presented at AAAI-19, CAMO (Collaborative rAnking via dynaMic rOuting) addresses this by capturing both word-level sequences and high-level topics through a hierarchical neural encoder and adversarial training.
Problem Statement
Existing CBR methods often struggle with two main issues:
- Semantic Hierarchy: Language is hierarchical—words form sentences where order matters, and sentences form topics where order is less critical. Most models treat documents either as unordered bags-of-words or as single long sequences, failing to capture this structure.
- Complexity vs. Generalization: Sophisticated models needed for deep text understanding frequently overfit on the sparse interaction data typical of recommendation tasks.
Methodology
CAMO introduces a three-layer content encoder designed to balance semantic depth with model robustness.
The Hierarchical Encoder
- Word2Vec Layer: Maps words to standard embeddings.
- Sent2Vec Layer: Employs a Gated Recurrent Unit (GRU) to capture intra-sentence word order, followed by an attention mechanism to generate sentence vectors.
- Doc2Vec Layer: Utilizes a Dynamic Routing protocol (inspired by Capsule Networks) to iteratively cluster sentences into "topics." This allows the model to dynamically prioritize relevant content and prune noise.
Adversarial Training
To mitigate overfitting, CAMO uses a minimax game strategy. An auxiliary "opponent" decoder attempts to reconstruct the original text from the document vector. The encoder is trained to minimize recommendation error while ensuring the learned vector is robust enough to support reconstruction, effectively regularizing the latent space.
Experimental Evaluation
The model was tested on three benchmarks: CiteULike (scientific abstracts), MovPlot1M, and MovPlot10M (movie plots).
Key Results
- Ranking Accuracy: CAMO consistently outperformed baselines like CMLST and CDL. On CiteULike, it achieved an AUC of 0.9540 and a MAP of 0.1623.
- Robustness: As shown in the Recall@50 metrics, CAMO maintains a significant lead over competitors across different datasets and negative sampling rates.
- Qualitative Insights: t-SNE visualizations indicated that CAMO produces cleaner genre-based clusters compared to models like CMLST, which often confuse items with overlapping vocabularies.
Discussion and Future Work
CAMO demonstrates that hierarchical modeling combined with adversarial regularization can significantly improve content-based recommendation. Future research directions include extending the framework to multi-modal data (images/video) and exploring probabilistic methods to further enhance resilience to noisy content.
Summary updated