CATA++: A Collaborative Dual Attentive Autoencoder Method for Recommending Scientific Articles

Meshal Alfarhood, Jianlin Cheng

Abstract

Recommender systems today have become an essential component of any commercial website. Collaborative filtering approaches, and Matrix Factorization (MF) techniques in particular, are widely used in recommender systems. However, the natural data sparsity problem limits their performance where users generally interact with very few items in the system. Consequently, multiple hybrid models were proposed recently to optimize MF performance by incorporating additional contextual information in its learning process. Although these models improve the recommendation quality, there are two primary aspects for further improvements: (1) multiple models focus only on some portion of the available contextual information and neglect other portions; (2) learning the feature space of the side contextual information needs to be further enhanced. In this paper, we introduce a Collaborative Dual Attentive Autoencoder (CATA++) for recommending scientific articles. CATA++ utilizes an article's content and learns its latent space via two parallel autoencoders. We employ the attention mechanism to capture the most related parts of information in order to make more relevant recommendations. Extensive experiments on three real-world datasets have shown that our dual-way learning strategy has significantly improved the MF performance in comparison with other state-of-the-art MF-based models using various experimental evaluations. The source code of our methods is available at: https://github.com/jianlin-cheng/CATA.

Introduction

In the domain of digital libraries and academic platforms, recommending relevant scientific articles is a critical but difficult task. The primary obstacle is extreme data sparsity: with millions of published papers, individual users typically interact with only a negligible fraction of the total corpus. Traditional Collaborative Filtering (CF) and Matrix Factorization (MF) methods, which rely on identifying patterns in user-item interaction matrices, often fail in these "cold-start" or sparse scenarios because the matrices are mostly empty.

To mitigate this, hybrid recommendation models have been developed to supplement interaction data with item metadata, such as abstracts and titles. However, existing deep learning-based hybrid models like Collaborative Deep Learning (CDL) or Collaborative Variational Autoencoder (CVAE) often treat all input features with equal importance. This approach fails to recognize that specific keywords or tags are significantly more indicative of an article's core topic than others. Furthermore, many models utilize only a subset of available metadata, ignoring valuable signals from user-generated tags or citation networks.

The CATA++ Framework

To address these limitations, we propose CATA++ (Collaborative Dual Attentive Autoencoder). This model integrates deep feature learning with Probabilistic Matrix Factorization (PMF) to handle implicit feedback. The architecture is designed to extract high-quality latent representations from heterogeneous data sources.

The CATA++ architecture featuring dual attentive autoencoders integrated with a Matrix Factorization framework.

The core innovations of CATA++ include:

  • Dual Autoencoders: The model employs two parallel neural networks. One processes textual data (titles and abstracts), while the second handles metadata such as tags and citation relationships.
  • Attention Mechanism: An attention layer, implemented via a softmax function, is inserted into the bottleneck of both autoencoders. This mechanism acts as a dynamic filter, forcing the model to prioritize the most informative features (words or tags) during the reconstruction process.
  • Integration with PMF: The compressed, attention-weighted latent representations from both networks are merged into a single vector. This vector serves as the Gaussian prior for the items' latent factors in the MF model, effectively guiding the recommendation engine with content-aware insights.

Experimental Setup

We evaluated CATA++ against several state-of-the-art baselines using three real-world datasets from CiteULike:

  1. Citeulike-a: 5,551 users and 16,980 articles (99.78% sparsity).
  2. Citeulike-t: 7,947 users and 25,975 articles (99.93% sparsity).
  3. Citeulike-2004-2007: The largest and most challenging dataset with 3,039 users and 210,137 articles (99.95% sparsity). This dataset lacks citation data, testing the model's robustness in extremely sparse environments.

Results and Discussion

CATA++ consistently outperformed all baseline models, including POP, CDL, CVAE, and the previous single-network CATA model, across various metrics such as Recall@K and normalized Discounted Cumulative Gain (nDCG@K).

Performance comparison of Recall@K across three CiteULike datasets, demonstrating the superior accuracy of CATA++.

The performance gains were most pronounced in the most sparse settings. For the Citeulike-2004-2007 dataset, CATA++ achieved a 58.36% improvement in Recall@10 and a 20.29% improvement in nDCG@10 over the strongest competing baseline. Qualitative analysis further supported these findings; in "cold-start" cases where a user had only one interaction, CATA++ successfully recommended semantically relevant articles, whereas other models often defaulted to generic, popular papers.

Key insights from our study include:

  • Noise Reduction: The attention mechanism effectively filters out noise in sparse bag-of-words representations, isolating the semantic core of an article more effectively than standard denoising or variational autoencoders.
  • Contextual Synergy: Ablation studies confirmed that the dual-network approach provides superior results compared to using either text or metadata alone, as these sources provide complementary signals.

Conclusion and Future Work

CATA++ demonstrates that combining dual-way feature learning with an attention mechanism significantly enhances the performance of recommendation systems for scientific articles. While the current model relies on Matrix Factorization, we acknowledge that the dot product used for scoring does not necessarily satisfy the triangle inequality. Future work will explore replacing the MF component with metric learning algorithms to more accurately map the geometric relationships between users and items in the latent space.

Summary updated

Abstract

Recommender systems today have become an essential component of any commercial website. Collaborative filtering approaches, and Matrix Factorization (MF) techniques in particular, are widely used in recommender systems. However, the natural data sparsity problem limits their performance where users generally interact with very few items in the system. Consequently, multiple hybrid models were proposed recently to optimize MF performance by incorporating additional contextual information in its learning process. Although these models improve the recommendation quality, there are two primary aspects for further improvements: (1) multiple models focus only on some portion of the available contextual information and neglect other portions; (2) learning the feature space of the side contextual information needs to be further enhanced. In this paper, we introduce a Collaborative Dual Attentive Autoencoder (CATA++) for recommending scientific articles. CATA++ utilizes an article's content and learns its latent space via two parallel autoencoders. We employ the attention mechanism to capture the most related parts of information in order to make more relevant recommendations. Extensive experiments on three real-world datasets have shown that our dual-way learning strategy has significantly improved the MF performance in comparison with other state-of-the-art MF-based models using various experimental evaluations. The source code of our methods is available at: https://github.com/jianlin-cheng/CATA.
BibTeX
@inproceedings{alfarhood2020cata,
  author = {Meshal Alfarhood and Jianlin Cheng},
  title = {{CATA++: A Collaborative Dual Attentive Autoencoder Method for Recommending Scientific Articles}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=aiP62n4PzU},
}
Figure