Aspect-Driven User Preference and News Representation Learning for News Recommendation

Wenpeng Lu, Rongyao Wang, Shoujin Wang, Xueping Peng, Hao Wu, Qian Zhang

TL;DR

Most news recommender systems rely on broad topic labels (like "Sports" or "Health"), which fail to capture a user's specific, fine-grained interests. This paper introduces the Aspect-driven News Recommender System (ANRS), which automatically extracts and leverages "aspects"—clusters of related words representing specific sub-topics—to build richer representations of both news articles and user preferences. By shifting from coarse topics to granular aspects, ANRS significantly improves personalized news recommendation accuracy.

What Problem Does This Paper Address?

In the realm of news recommendation, understanding what a user wants to read is paramount. Historically, state-of-the-art models have relied on coarse-grained "topic" information to match users with articles. However, the authors identify three major flaws with this approach:

  1. Topics are too broad: An article tagged with the topic "Auto" might be specifically about "Ford" and "Australia."
  2. User preferences are nuanced: A user might love reading about "Ford" but have zero interest in other "Auto" news. Recommending all car-related news degrades their experience.
  3. Labels are expensive: Thousands of news articles are generated daily. Manually annotating them with accurate topic labels is time-consuming and often impossible, limiting the real-world applicability of topic-dependent models.

To bridge this gap, the authors propose using aspects—fine-grained semantic details expressed by sets of related words (e.g., "beef, pasta, tomato" instead of just "food"). Crucially, these aspects can be extracted automatically from the text, bypassing the need for manual labeling.

Methods

The authors propose ANRS, a neural architecture consisting of three main modules:

  1. News Aspect-level Encoder: This module processes the news title, abstract, and category using Convolutional Neural Networks (CNNs) and word-level attention to capture general features. The novel addition is the aspect-level feature extractor. Inspired by autoencoders, it uses an attention mechanism to identify key aspect words in the text. It then reconstructs the news embedding via a linear combination of learnable aspect embeddings (initialized using K-means clustering of word embeddings). A custom loss function ensures these aspect embeddings remain diverse and unique.
  2. User Aspect-level Encoder: To understand the user, this module looks at their reading history. It applies a sentence-level attention mechanism over the aspect-level representations of the user's previously browsed news, aggregating them into a single, personalized preference vector.
  3. Click Predictor: Finally, the system calculates the probability of a user clicking a candidate article by taking the inner product of the user's preference representation and the candidate news's representation.

Datasets

The model was trained and evaluated on MIND (Microsoft News Dataset), a massive, real-world dataset collected from anonymous behavior logs on the Microsoft News website. The authors utilized both available versions:

  • MIND-small: 65,238 news articles and 94,057 users.
  • MIND-large: 161,013 news articles and 1,000,000 users (containing over 24 million click logs).

Key Insights and Ideas

  • Granularity wins: Fine-grained aspect information is demonstrably more powerful than traditional coarse topic labels. Capturing the specific entities or sub-topics a user cares about leads to much better click prediction.
  • Unsupervised extraction is highly effective: You don't need expensive manual labels to get good aspects. The autoencoder-like reconstruction loss effectively forces the model to learn meaningful, diverse aspect clusters automatically (e.g., separating "restaurant" and "America" as distinct aspects from a generic "food" article).
  • The "Goldilocks" number of aspects: The authors found that 40 aspects was the optimal hyperparameter for this dataset. Too few aspects fail to capture nuance, while too many introduce noise and cause overfitting.
  • Titles are king: Ablation studies on the input data showed that news titles provide more efficient and impactful signals for aspect extraction than the longer abstracts.

Key Results

ANRS was compared against a robust lineup of baselines, ranging from traditional matrix factorization (BiasMF, FM) to state-of-the-art neural models (NRMS, TANR, LSTUR).

  • Superior Accuracy: ANRS outperformed all baselines. On the MIND-large dataset, ANRS achieved an AUC of 0.6826 and an MRR of 0.3350, beating the top-performing baselines NRMS (AUC 0.6701) and TANR (AUC 0.6611).
  • Ablation Success: To prove the value of the aspect components, the authors tested a stripped-down version of the model ($\text{ANRS}^{-a}$) that removed the aspect-level feature extractors. Performance dropped significantly (AUC fell from 0.6826 to 0.6761), confirming that the aspect-driven approach is the primary driver of the model's success.

Limitations and Future Work

  • Ignored Body Text: Currently, ANRS only uses the title, abstract, and category of the news. The authors acknowledge that the full body text of the articles likely contains even richer aspect information that is currently being left on the table.
  • Architecture Exploration: While CNNs and standard attention mechanisms worked well here, the authors suggest that exploring more advanced or effective neural architectures (like Transformers) could capture aspect information even more accurately.
  • Future Directions: The team plans to incorporate full news body content into the modeling process and further refine the neural architecture for aspect extraction in future iterations.

Summary updated

Abstract

Intelligent human-device interfaces play key roles in fully automated vehicles (FAVs), ensuring smooth interactions and improving the driving experience. Listening to news is a popular method of relaxing during a journey; as a result, travelers require automatic recommendations of preferred news programs. Most existing news recommender systems usually learn topic-level representations of users and news for recommendations while neglecting to learn more informative aspect-level features, resulting in limited recommendation performance. To bridge this significant gap, we propose a novel Aspect-driven News Recommender System (ANRS) built on aspect-level user preferences and news representation learning. In ANRS, a news aspect-level encoder and a user aspect-level encoder are devised to learn the fine-grained aspect-level representations of users’ preferences and news characteristics respectively. These representations are subsequently fed into a click predictor to predict the probability of a given user clicking on the candidate news item. Extensive experiments demonstrate the superiority of our method over state-of-the-art baseline methods.
BibTeX
@article{lu2022aspect,
  author = {Wenpeng Lu and Rongyao Wang and Shoujin Wang and Xueping Peng and Hao Wu and Qian Zhang},
  title = {{Aspect-Driven User Preference and News Representation Learning for News Recommendation}},
  journal = {IEEE Trans. Intell. Transp. Syst.},
  year = {2022},
  doi = {10.1109/tits.2022.3182568},
}

Loading related content...

Figure