Counterfactual Maximum Likelihood Estimation for Training Deep Networks

Xinyi Wang, Wenhu Chen, Michael Saxon, William Yang Wang

Abstract

Although deep learning models have driven state-of-the-art performance on a wide array of tasks, they are prone to spurious correlations that should not be learned as predictive clues. To mitigate this problem, we propose a causality-based training framework to reduce the spurious correlations caused by observed confounders. We give theoretical analysis on the underlying general Structural Causal Model (SCM) and propose to perform Maximum Likelihood Estimation (MLE) on the interventional distribution instead of the observational distribution, namely Counterfactual Maximum Likelihood Estimation (CMLE). As the interventional distribution, in general, is hidden from the observational data, we then derive two different upper bounds of the expected negative log-likelihood and propose two general algorithms, Implicit CMLE and Explicit CMLE, for causal predictions of deep learning models using observational data. We conduct experiments on both simulated data and two real-world tasks: Natural Language Inference (NLI) and Image Captioning. The results show that CMLE methods outperform the regular MLE method in terms of out-of-domain generalization performance and reducing spurious correlations, while maintaining comparable performance on the regular evaluations.

Introduction: The Challenge of Spurious Correlations

Deep learning models are notorious for being "right for the wrong reasons." Because they are typically trained using standard Maximum Likelihood Estimation (MLE) on observational data, they often latch onto spurious correlations—statistical shortcuts that happen to be true in the training set but are not actually causal. For example, a computer vision model might only recognize a "cow" when it appears against a green pasture, failing if the cow is on a beach. In Natural Language Inference (NLI), models often predict relationships by looking at just one sentence, ignoring the actual logic between the premise and hypothesis.

In image captioning, this manifests as "object hallucination," where models predict objects (like a toilet in a bathroom) simply because they frequently co-occur in the dataset, even if they are absent from the specific image. The authors argue that the current associative learning framework is the culprit and propose moving toward learning causal mechanisms.

Methodology: From Observation to Intervention

The core of the proposed approach is Counterfactual Maximum Likelihood Estimation (CMLE). Instead of training on the observational distribution $p^O(X, Y, T)$, which contains confounding bias, the framework aims to train on an interventional distribution $p^I(X, Y, T)$. This represents a hypothetical world where the link between input features and labels is "broken" to remove bias.

Causal graph illustrating the transition from observational data (a) to the interventional distribution (d) used in CMLE.

Since data from this interventional world is not directly available, the authors derive two mathematical upper bounds for the interventional loss that can be optimized using observational data:

  1. Implicit CMLE: This method focuses on internal representations. It utilizes a regularizer based on the Wasserstein distance to ensure that latent representations are balanced across different labels, preventing the model from using "shortcut" features in the representation space.
  2. Explicit CMLE: This method uses a secondary model to explicitly generate "counterfactual" examples during training. It asks: "If the label were different, what would the input look like?" and trains the main model to handle these generated scenarios.

Experimental Evaluation

The authors validated CMLE across three distinct environments: simulated data, Natural Language Inference (NLI), and Image Captioning.

Natural Language Inference (NLI)

Using the standard SNLI and MNLI datasets for training, the models were evaluated on ANLI (Adversarial NLI), a dataset designed to trip up models relying on spurious correlations. CMLE improved accuracy by 2.9% relative over standard MLE and demonstrated significantly higher "labeling consistency," indicating more logically sound hypothesis generation.

Image Captioning

Evaluated on the MSCOCO 2014 dataset, CMLE significantly lowered CHAIR scores, a key metric for object hallucination. This indicates that the models were less likely to claim they saw objects that were not present in the image. Furthermore, in user studies, humans preferred CMLE-generated captions over standard MLE captions by over 10%, citing better faithfulness and informativeness.

Conclusion and Future Work

CMLE successfully translates "do-calculus" from causal inference into a practical loss function for deep learning. Notably, it requires no manual human intervention or data augmentation, as the process is fully automated. While the framework currently relies on the Strong Ignorability Assumption (assuming no unobserved confounders), it demonstrates that reducing spurious correlations is possible without sacrificing in-domain performance. Future work may explore incorporating unobserved confounders to make the framework even more robust for complex real-world scenarios.

Summary updated

Abstract

Although deep learning models have driven state-of-the-art performance on a wide array of tasks, they are prone to spurious correlations that should not be learned as predictive clues. To mitigate this problem, we propose a causality-based training framework to reduce the spurious correlations caused by observed confounders. We give theoretical analysis on the underlying general Structural Causal Model (SCM) and propose to perform Maximum Likelihood Estimation (MLE) on the interventional distribution instead of the observational distribution, namely Counterfactual Maximum Likelihood Estimation (CMLE). As the interventional distribution, in general, is hidden from the observational data, we then derive two different upper bounds of the expected negative log-likelihood and propose two general algorithms, Implicit CMLE and Explicit CMLE, for causal predictions of deep learning models using observational data. We conduct experiments on both simulated data and two real-world tasks: Natural Language Inference (NLI) and Image Captioning. The results show that CMLE methods outperform the regular MLE method in terms of out-of-domain generalization performance and reducing spurious correlations, while maintaining comparable performance on the regular evaluations.
BibTeX
@inproceedings{wang2021counterfactual,
  author = {Xinyi Wang and Wenhu Chen and Michael Saxon and William Yang Wang},
  title = {{Counterfactual Maximum Likelihood Estimation for Training Deep Networks}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=o6s1b_-nDOE},
}
Figure