Optimal Transport on Categorical Data for Conterfactuals Using Compositional Data and Dirichlet Transport

Agathe Fernandes Machado, Ewen Gallic, Arthur CharpentierIJCAI 2025 · DOI

TL;DR

This paper introduces a novel method for generating counterfactuals for categorical data by transforming categories into "compositional data" and applying optimal transport on the probability simplex. This approach addresses the challenge of applying existing optimal transport methods, which struggle with the discrete and unordered nature of categorical variables, enabling more robust fairness assessments and causal inference in real-world scenarios.

What Problem Does This Paper Address?

Counterfactual analysis is crucial for understanding causality and fairness, particularly in machine learning. It involves reasoning about "what if" scenarios, such as how a decision might change if a sensitive attribute (like gender or race) were different. While optimal transport (OT) has shown promise for continuous data, it faces significant hurdles when dealing with categorical variables. Unlike continuous variables with natural distance metrics, categorical variables lack an inherent ordering or distance, making it difficult to define meaningful "transport" between different categories. This limitation hinders the application of counterfactual fairness and other causal inference techniques to datasets with predominantly categorical features.

Methods

The core of this paper lies in transforming categorical variables into a format amenable to OT. The authors propose:

  1. Categorical to Compositional Data Transformation: Each categorical variable with $d$ categories is converted into a $d$-dimensional vector representing probabilities or proportions, residing on the probability simplex ($S_d$). This is achieved using probabilistic classifiers, such as Multinomial Logistic Regression (MLR), which predict the likelihood of each category given other features. The output is then normalized to sum to 1, forming a compositional vector.
  2. Optimal Transport on the Simplex: Once variables are represented as compositions on the simplex, the authors explore two main OT strategies:
  3. Gaussian OT via Euclidean Representation: They map the compositional data from the simplex ($S_d$) to a lower-dimensional Euclidean space ($R^{d-1}$) using isomorphisms like the additive log-ratio (alr), centered log-ratio (clr), or isometric log-ratio (ilr) transforms. Standard Gaussian OT is then applied in this Euclidean space, and the results are mapped back to the simplex.
  4. Dirichlet Transport: This approach directly operates on the simplex using a tailored cost function (related to L-divergence/cross-entropy) that respects the geometry of the simplex. This method avoids the need for intermediate Euclidean representations and is particularly relevant when dealing with distributions that naturally live on the simplex, like Dirichlet distributions.

Datasets

The paper uses two well-known benchmark datasets for evaluation:

  • German Credit Dataset: Used to assess counterfactuals for the "Purpose" of a loan, considering "Sex" as the sensitive attribute.
  • Adult Dataset: Used to analyze counterfactuals for "Marital Status," considering "Sex" as the sensitive attribute.

The authors note that their approach is applicable to any context requiring counterfactual analysis on categorical data and does not require a pre-defined structural causal model (SCM).

Key Insights and Ideas

  • Bridging the Categorical Gap: The most significant insight is the successful transformation of discrete, unordered categorical variables into a continuous, geometrically structured space (the simplex) where OT can be meaningfully applied. This opens up counterfactual analysis to a much wider range of real-world datasets.
  • Compositional Data as a Natural Representation: Representing categories as proportions or probabilities on the simplex aligns well with the idea of "compositional data" and naturally captures uncertainty and relationships between categories.
  • Simplex Geometry Matters: The paper highlights the importance of respecting the geometric properties of the simplex, leading to the exploration of "Dirichlet transport" as a more theoretically grounded approach than simply mapping to Euclidean space.
  • No Arbitrary Ordering Needed: A key advantage is that the method does not require imposing an arbitrary order on categorical labels, which is a common limitation in other approaches.

Key Results

The paper demonstrates the applicability of its methods through empirical illustrations on the German Credit and Adult datasets. The results show that:

  • Plausible Counterfactuals: The generated counterfactuals for loan purpose and marital status, when changing the sensitive attribute (sex), produce plausible shifts in the predicted categories. For instance, a "separated" woman's counterfactual marital status shifts towards "married" when considering the male population's distribution.
  • Model Sensitivity: Different underlying models (GAM-MLR, Random Forest, Gradient Boosting) used for the initial categorical-to-compositional transformation yield slightly different counterfactuals, highlighting the importance of the predictive model's quality.
  • Visualizations: Figures in the paper effectively visualize the transport process on the simplex, showing how points (representing compositions) are mapped from one distribution to another.

Limitations and Future Work

The authors acknowledge several limitations:

  • Computational Intensity: Optimal transport computations, especially on high-dimensional simplices, can be computationally intensive, posing challenges for very large datasets.
  • Classifier Vulnerability: The initial step of using a probabilistic classifier to create compositions can be a vulnerability. Biases in the classifier or issues with scarce categories (requiring grouping) can impact the quality of the counterfactuals.

Future work could focus on developing more efficient OT algorithms for the simplex, exploring robust methods for handling scarce categories, and further integrating this approach within full structural causal models for more comprehensive causal inference.

Summary updated

BibTeX
@inproceedings{machado2025optimal,
  author = {Agathe Fernandes Machado and Ewen Gallic and Arthur Charpentier},
  title = {{Optimal Transport on Categorical Data for Conterfactuals Using Compositional Data and Dirichlet Transport}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/572},
}
Figure