CAMO: A Collaborative Ranking Method for Content Based Recommendation

Chengwei Wang, Tengfei Zhou, Chen Chen 0043, Tianlei Hu, Gang Chen 0001

Introduction

In recommender systems, content-based recommendation (CBR) is vital when user feedback is sparse. However, effectively extracting features from text remains a challenge: simple models often miss linguistic nuances, while complex neural networks are prone to overfitting. Presented at AAAI-19, CAMO (Collaborative rAnking via dynaMic rOuting) addresses this by capturing both word-level sequences and high-level topics through a hierarchical neural encoder and adversarial training.

Problem Statement

Existing CBR methods often struggle with two main issues:

  1. Semantic Hierarchy: Language is hierarchical—words form sentences where order matters, and sentences form topics where order is less critical. Most models treat documents either as unordered bags-of-words or as single long sequences, failing to capture this structure.
  2. Complexity vs. Generalization: Sophisticated models needed for deep text understanding frequently overfit on the sparse interaction data typical of recommendation tasks.

Methodology

CAMO introduces a three-layer content encoder designed to balance semantic depth with model robustness.

The CAMO Content Encoder architecture, showing the hierarchical transition from Word2Vec to Doc2Vec.

The Hierarchical Encoder

  • Word2Vec Layer: Maps words to standard embeddings.
  • Sent2Vec Layer: Employs a Gated Recurrent Unit (GRU) to capture intra-sentence word order, followed by an attention mechanism to generate sentence vectors.
  • Doc2Vec Layer: Utilizes a Dynamic Routing protocol (inspired by Capsule Networks) to iteratively cluster sentences into "topics." This allows the model to dynamically prioritize relevant content and prune noise.

Adversarial Training

To mitigate overfitting, CAMO uses a minimax game strategy. An auxiliary "opponent" decoder attempts to reconstruct the original text from the document vector. The encoder is trained to minimize recommendation error while ensuring the learned vector is robust enough to support reconstruction, effectively regularizing the latent space.

Experimental Evaluation

The model was tested on three benchmarks: CiteULike (scientific abstracts), MovPlot1M, and MovPlot10M (movie plots).

Performance comparison (Recall@50) of CAMO against CMLST and CDL across three datasets with varying negative sampling (NS) rates.

Key Results

  • Ranking Accuracy: CAMO consistently outperformed baselines like CMLST and CDL. On CiteULike, it achieved an AUC of 0.9540 and a MAP of 0.1623.
  • Robustness: As shown in the Recall@50 metrics, CAMO maintains a significant lead over competitors across different datasets and negative sampling rates.
  • Qualitative Insights: t-SNE visualizations indicated that CAMO produces cleaner genre-based clusters compared to models like CMLST, which often confuse items with overlapping vocabularies.

Discussion and Future Work

CAMO demonstrates that hierarchical modeling combined with adversarial regularization can significantly improve content-based recommendation. Future research directions include extending the framework to multi-modal data (images/video) and exploring probabilistic methods to further enhance resilience to noisy content.

Summary updated

Abstract

In real-world recommendation tasks, feedback data are usually sparse. Therefore, a recommender’s performance is often determined by how much information that it can extract from textual contents. However, current methods do not make full use of the semantic information. They encode the textual contents either by “bag-of-words” technique or Recurrent Neural Network (RNN). The former neglects the order of words while the latter ignores the fact that textual contents can contain multiple topics. Besides, there exists a dilemma in designing a recommender. On the one hand, we shall use a sophisticated model to exploit every drop of information in item contents; on the other hand, we shall adopt a simple model to prevent itself from over-fitting when facing the sparse feedbacks. To fill the gaps, we propose a recommender named CAMO 1. CAMO employs a multi-layer content encoder for simultaneously capturing the semantic information of multitopic and word order. Moreover, CAMO makes use of adversarial training to prevent the complex encoder from overfitting. Extensive empirical studies show that CAMO outperforms state-of-the-art methods in predicting users’ preferences.
BibTeX
@inproceedings{wang2019camo,
  author = {Chengwei Wang and Tengfei Zhou and Chen Chen and Tianlei Hu and Gang Chen},
  title = {{CAMO: A Collaborative Ranking Method for Content Based Recommendation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33015224},
}
Figure