Information Theoretic Counterfactual Learning from Missing-Not-At-Random Feedback

Zifeng Wang, Xi Chen, Rui Wen, Shao-Lun Huang, Ercan E. Kuruoglu, Yefeng Zheng

Recommender systems often suffer from "policy bias" because they train on Missing-Not-At-Random (MNAR) data—users only rate items they choose to interact with. Traditionally, fixing this requires expensive Randomized Controlled Trials (RCTs). This paper introduces the Counterfactual Variational Information Bottleneck (CVIB), an information-theoretic framework that balances learning between observed and unobserved feedback, significantly improving model generalization without needing any costly RCT data.

The Challenge of Missing-Not-At-Random Data

In the real world, recommender systems train on logged feedback (like clicks or ratings). This data is Missing-Not-At-Random (MNAR): users are much more likely to interact with items they already like. If a model trains naively on this data, it suffers from "policy bias," overestimating the click-through rate and reinforcing a narrow bubble of recommendations.

Factual vs. Counterfactual Feedback

To debias these models, researchers typically rely on Missing-At-Random (MAR) data collected via Randomized Controlled Trials (RCTs)—literally forcing the system to recommend random items to users to see what happens. Methods like Inverse Propensity Scoring (IPS) use this MAR data to correct the bias. However, running RCTs is incredibly expensive, degrades the user experience, and is often impractical at scale. This paper tackles the challenge of debiasing recommender systems without relying on RCTs.

The Counterfactual Variational Information Bottleneck (CVIB)

The authors propose the Counterfactual Variational Information Bottleneck (CVIB). It builds on the Information Bottleneck (IB) principle, which encourages a learned representation (embedding) to be highly predictive of the target task while compressing away irrelevant input noise.

CVIB Architecture

To adapt IB for counterfactual learning, the authors split the mutual information objective into two domains: factual (observed events) and counterfactual (unobserved events). Because the true labels for counterfactuals are unknown, they derive a clever, tractable loss function with four main components:

  1. Sufficiency: A standard supervised loss (cross-entropy) on the observed factual data.
  2. Balancing: A contrastive regularization term that forces the model's predictions on factual and counterfactual domains to remain close, preventing it from wildly overestimating observed events.
  3. Penalty: A confidence penalty on the factual outputs to prevent the model from becoming overly certain about the biased training data.
  4. Minimality: A regularization term (acting like an L2 norm) that strips away information related to the "policy bias" (the nuisance variable dictating whether an item was shown to the user in the first place).

Experimental Setup

To properly evaluate debiasing methods, datasets must contain both biased training data and unbiased testing data. The authors use two standard benchmarks:

  • Yahoo! R3 Dataset: Contains over 300K self-selected user-song ratings (MNAR) for training, and an additional MAR test set where 5,400 users were asked to rate 10 randomly displayed songs.
  • Coat Shopping Dataset: Contains 290 users rating 24 self-selected items (MNAR) for training, and a MAR test set where users rated 16 uniformly displayed items.

Key Insights

  • Minimality naturally fights bias: The authors mathematically prove that the "minimality" term in the Information Bottleneck naturally makes embeddings insensitive to policy bias. By forcing the embedding to forget unnecessary input details, it forgets the bias that caused the item to be displayed in the first place.
  • You don't need true labels to balance the model: The genius of CVIB is that it doesn't try to guess the exact missing ratings. Instead, it minimizes the distance between the distributions of factual and counterfactual predictions. This prevents the model from overfitting to the "happy path" of observed clicks.
  • Ranking quality > Absolute error: The deep learning variant of their model (NCF-CVIB) actually produced worse Mean Squared Error (MSE) than some baselines, but vastly superior ranking metrics (AUC/nDCG). In real recommenders, correctly ranking a positive item above a negative one is far more important than perfectly predicting the exact rating value.

Results

The authors applied CVIB to both shallow (Matrix Factorization - MF) and deep (Neural Collaborative Filtering - NCF) architectures. They compared it against standard models and popular RCT-dependent debiasing methods (IPS, SNIPS, DR, DRJL).

Remarkably, CVIB outperformed baselines that had access to RCT data, despite CVIB using zero RCT data itself:

  • On the Coat Dataset: MF-CVIB achieved an AUC of 0.7218 (vs. the best baseline at 0.7156) and an nDCG@5 of 0.663 (vs. 0.633).
  • On the Yahoo Dataset: MF-CVIB achieved an AUC of 0.7198 (vs. the best baseline at 0.6892) and an nDCG@5 of 0.734 (vs. 0.659).
  • Deep models (NCF) showed similar consistent improvements in ranking metrics across both datasets.

Limitations and Future Directions

  • Hyperparameter Sensitivity: The model's success relies heavily on the balancing hyperparameter ($\alpha$). Tuning this parameter is crucial and its optimal value varies significantly between datasets.
  • MSE Trade-offs: The deep version of CVIB can overestimate absolute outcomes, leading to worse MSE despite better ranking.
  • Future Work: By properly accounting for unobserved events, this information-theoretic approach has strong potential to improve not just the accuracy of recommender systems, but also their diversity and fairness.

Summary updated

Abstract

Counterfactual learning for dealing with missing-not-at-random data (MNAR) is an intriguing topic in the recommendation literature, since MNAR data are ubiquitous in modern recommender systems. Instead, missing-at-random (MAR) data, namely randomized controlled trials (RCTs), are usually required by most previous counterfactual learning methods. However, the execution of RCTs is extraordinarily expensive in practice. To circumvent the use of RCTs, we build an information theoretic counterfactual variational information bottleneck (CVIB), as an alternative for debiasing learning without RCTs. By separating the task-aware mutual information term in the original information bottleneck Lagrangian into factual and counterfactual parts, we derive a contrastive information loss and an additional output confidence penalty, which facilitates balanced learning between the factual and counterfactual domains. Empirical evaluation on real-world datasets shows that our CVIB significantly enhances both shallow and deep models, which sheds light on counterfactual learning in recommendation that goes beyond RCTs.
BibTeX
@inproceedings{wang2020information,
  author = {Zifeng Wang and Xi Chen and Rui Wen and Shao-Lun Huang and Ercan E. Kuruoglu and Yefeng Zheng},
  title = {{Information Theoretic Counterfactual Learning from Missing-Not-At-Random Feedback}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=UOCx8KmVCdVZ},
}
Figure