Introduction
In the current machine learning landscape, most models operate under the "i.i.d. assumption," which presumes that training and testing data are drawn from the same distribution. However, real-world applications—ranging from medical diagnostics to financial forecasting—frequently encounter distributional shifts. While Invariant Risk Minimization (IRM) has emerged as a powerful technique to learn features that remain stable across different environments, it primarily provides point predictions without rigorous uncertainty quantification.
This paper introduces Adaptive Conformal Inference for Invariant Representation (ACIR), a framework designed to bridge this gap. By combining the robustness of invariant learners with the statistical guarantees of conformal inference, ACIR generates prediction intervals that are both reliable and efficient, even when facing out-of-distribution data.
The ACIR Framework
Standard Conformal Inference typically requires data to be exchangeable, an assumption that is violated when environments shift. ACIR overcomes this by utilizing a two-pronged approach:
- Invariance Assessment: The authors introduce a diagnostic statistic,
Inv(Φ), to measure how effectively a learned representation achieves invariance across environments. This ensures the underlying model is a suitable candidate for invariant inference. - Weighted Conformity Scores: The core of the ACIR algorithm is a novel weighting scheme. When a test point is processed, the algorithm evaluates its latent representation's similarity (based on mean and variance) to known training environments. Calibration data from environments most similar to the test point are assigned higher weights. This allows the prediction interval to adapt locally to the specific environment of the test point, rather than relying on a global, overly conservative estimate.
Experimental Setup
The method was evaluated using both synthetic and real-world datasets:
- Synthetic Structural Equation Model (SEM): Designed to simulate causal and non-causal relationships across four settings: Partially-observed Homoskedastic (POU), Partially-observed Heteroskedastic (PEU), Fully-observed Homoskedastic (FOU), and Fully-observed Heteroskedastic (FEU).
- Financial Market Data: A dataset of U.S. stock market indicators (2014–2018), where each year represents a distinct environment, testing the model's ability to handle temporal shifts.
Key Results and Discussion
The primary advantages of ACIR are its reliability in maintaining coverage and its efficiency in producing narrow intervals.
As illustrated in Figure 1, ACIR consistently meets the target coverage requirements across all four synthetic settings. While standard Split Conformal (SC) methods often "over-cover"—producing unnecessarily wide intervals to ensure safety—ACIR remains closer to the nominal level, indicating a more precise calibration of uncertainty.
Figure 4 highlights the efficiency gains of the ACIR approach. In settings characterized by heteroskedasticity (PEU and FEU), where noise levels vary across environments, ACIR produces significantly narrower intervals than SC methods. This efficiency was also reflected in the real-world financial data, where ACIR achieved a coverage of ~79.5% (against an 80% target) with an average interval length of 17.4, compared to 21.5 for the standard method.
Conclusion and Future Work
ACIR demonstrates that invariance can serve as a robust foundation for uncertainty quantification. By moving from marginal coverage to a more localized, conditional-style coverage through similarity-based weighting, the framework provides a practical tool for risk management in shifting environments.
Future research directions include expanding the weighting scheme beyond first and second moments to capture multi-modal or skewed distributions and more tightly integrating the invariance penalty directly into the interval calibration process.
Summary updated