Introduction
Deep learning models frequently rely on Maximum Likelihood Estimation (MLE), which essentially optimizes for the most likely label given the training data. However, MLE is purely associative. If a dataset contains inherent biases—for example, if most "dog" photos are taken on grass—the model might learn that "grass" is a predictive feature for "dog."
These learned shortcuts, known as spurious correlations, cause models to fail when tested on out-of-domain data (e.g., a dog on a beach). This phenomenon is also prevalent in Natural Language Processing (NLP), where models might predict the relationship between two sentences by looking at just one, ignoring the actual logical relationship. To address this, this paper introduces Counterfactual Maximum Likelihood Estimation (CMLE), a training framework grounded in causal inference that forces models to ignore these shortcuts and focus on true underlying mechanisms.
Methodology: From Observation to Intervention
The authors propose a framework based on Structural Causal Models (SCM). They treat the input ($X$) as containing potential confounders and the label or constraint ($T$) as a "treatment." The ultimate goal is to estimate the interventional distribution—a hypothetical state of the world where the spurious link between the confounder and the label is broken.
Because we typically only have access to observational data, we cannot directly observe this counterfactual world. However, the authors demonstrate that we can mathematically bound the error of our predictions using the data we do have. By treating dataset bias as a form of "selection bias" (akin to estimating Individual Treatment Effects in medical research), they derive two mathematical upper bounds to approximate the hidden interventional state:
- Implicit CMLE: This method learns a balanced representation of the input. It uses the Wasserstein distance (a measure of how different two distributions are) as a regularizer. This forces the model to ensure that its internal extracted features look similar regardless of the label, effectively "bleaching out" the influence of confounders.
- Explicit CMLE: This approach is more direct. The model explicitly generates counterfactual examples during training. For instance, if the model sees a "factual" image-caption pair, Explicit CMLE generates a "counterfactual" version to test how the model's internal logic holds up. It uses a secondary model to approximate the interventional distribution and a Gumbel-Softmax trick to keep the process differentiable.
Experimental Setup
The researchers evaluated CMLE across three distinct environments to test its robustness and generalizability:
- Simulated Data: A synthetic dataset where the causal ground truth is known, allowing for precise measurement of how well the model captures the "true" mechanism.
- Natural Language Inference (NLI): The models were trained on SNLI and MNLI, and tested on the "hard" ANLI (Adversarial NLI) dataset to evaluate out-of-domain generalization.
- Image Captioning: The MSCOCO dataset (Karpathy split) was used to test "object hallucination"—the tendency of vision-language models to mention objects that aren't in the image simply because they frequently appear in similar training scenes.
Key Results
CMLE successfully improves out-of-domain performance while keeping standard accuracy comparable to vanilla MLE, demonstrating robustness without degradation.
- NLI Generalization: On the adversarial ANLI dataset, CMLE improved accuracy by 2.9% over standard MLE.
- Reduced Hallucination: In image captioning, CMLE significantly lowered the CHAIR score (a standard metric for object hallucination), meaning the models were much less likely to "see" objects that weren't actually there.
- Human Preference: Human evaluators preferred CMLE-generated captions over standard MLE baselines by more than 10%, specifically citing better faithfulness to the actual image.
- Explicit vs. Implicit: Explicit CMLE generally performed better than Implicit CMLE because it directly optimizes the counterfactual likelihood rather than just balancing representations.
Limitations and Future Work
The primary limitation of this approach is the Strong Ignorability Assumption. The authors assume that all confounders are "observed" within the input $X$. In real-world scenarios, there are often "unobserved confounders"—hidden factors not captured in the data—that influence both the input and the label.
Future work should focus on expanding the SCM to account for these unobserved variables, perhaps by modeling the inherent biases of the human annotators who created the datasets. Additionally, while the counterfactual generation in Explicit CMLE is automated, it is not perfect and can occasionally introduce its own artifacts into the training process.
Summary updated