This paper introduces NeMig, a specialized bilingual news dataset designed to move the field of news recommendation beyond simple click-prediction accuracy and toward understanding complex societal effects like filter bubbles and political polarization.
TL;DR
NeMig is a new bilingual (German and English) dataset focused on the controversial topic of migration, combining news articles with rich annotations (sentiment, political leaning, subtopics) and detailed user socio-demographic and political data. By providing domain-specific knowledge graphs and explicit user feedback, it enables researchers to analyze how recommendation algorithms influence opinion formation and reinforce biases in a way that standard, general-purpose datasets cannot.
What Problem Does This Paper Address?
Most modern news recommendation research relies on massive, general-purpose datasets (like Microsoft’s MIND) that focus primarily on "click-through rate" (CTR) optimization. While effective for engagement, this approach ignores the fundamental role media plays in shaping public worldview, especially regarding sensitive or polarizing topics.
Existing resources typically suffer from two gaps:
- Recommendation datasets are usually monolingual, cover "soft" topics (sports, entertainment), and lack deep information about the users' own political leanings or the articles' ideological slants.
- Media analysis datasets (used for bias or fake news detection) often have rich annotations but lack the user interaction data required to train or evaluate recommendation algorithms.
NeMig bridges this gap by providing a "thick" dataset centered on a single, highly polarizing topic—migration—allowing for interdisciplinary research into how algorithms interact with human political psychology.
Methods
The authors developed a comprehensive pipeline for data collection, annotation, and knowledge graph construction:
- Data Collection: They crawled over 18,000 articles from 40 German and 45 US news outlets, carefully selected to span the entire political spectrum.
- Multi-faceted Annotation:
- Sentiment: Articles were classified as positive, neutral, or negative using a multilingual XLM-R model.
- Sub-topics: Neural topic modeling identified specific themes (e.g., "border control," "Russian invasion of Ukraine").
- Named Entity Recognition (NER) & Linking: Entities (people, places, orgs) were extracted and disambiguated by linking them to Wikidata QIDs.
- Knowledge Graph (KG) Construction: The authors built NeMigKG, representing articles, entities, and metadata as nodes. They created four variants of the KG, including versions enriched with up to 2-hop neighbors from Wikidata to provide external context.
- User Study: Unlike datasets that scrape anonymous logs, the authors conducted controlled online studies with ~3,000 users in both Germany and the US. They collected explicit feedback (rating articles) alongside detailed demographics, political attitudes, and personality traits.
Datasets
The primary contribution is the NeMig dataset itself, which includes:
- German Corpus: 7,346 filtered articles (2019–2021).
- English Corpus: 10,814 filtered articles (2021–2022).
- User Data: Demographics and political profiles for ~6,000 total users.
- NeMigKG: A domain-specific knowledge graph available in base, entity-only, and Wikidata-enriched versions.
Key Insights and Ideas
- Knowledge-Awareness Wins on Small Data: In their benchmarks, knowledge-aware models like DKN and CAUM generally outperformed standard neural recommenders. This suggests that when training data is relatively small (thousands of articles rather than millions), injecting external knowledge via KGs is a powerful way to improve representations.
- The "1-Hop" Sweet Spot: When enriching KGs with external data from Wikidata, the authors found that adding 1-hop neighbors improved diversity and performance, but going to 2-hops often "diluted" the domain-specific signal, leading to worse results.
- Diversity vs. Personalization: The study confirmed that users tend to prefer news that aligns with their existing topical interests and political leanings rather than their sentiment preferences.
- Secondary Goals Don't Kill Accuracy: Experiments with SentiDebias showed that it's possible to optimize for sentiment diversity without significantly sacrificing standard recommendation accuracy.
Key Results
The authors benchmarked several state-of-the-art models (NRMS, NAML, DKN, etc.) on NeMig:
- Content Personalization: DKN and CAUM achieved the best results, with DKN reaching an AUC of 58.73 on the German dataset.
- Diversity: SentiDebias successfully provided the most sentiment-diverse recommendations. NAML was the most effective at providing diversity across topical categories.
- Political Bias: Interestingly, all benchmarked recommenders performed nearly identically regarding political diversification, likely because none were explicitly designed to counter political "filter bubbles."
Limitations and Future Work
- Scale: While "thick" in features, the user base (3K per language) is small compared to industry-scale datasets. The authors suggest using this real-world data as a seed to generate larger synthetic user datasets.
- Copyright Constraints: Due to legal restrictions, the full body text of the news articles is only available upon request for research, though all other features and metadata are freely available on GitHub.
- Future Directions: The authors aim to use NeMig to study whether recommendation algorithms actively amplify existing user biases over time, rather than just reflecting them.
Summary updated