Enhancing Classification Models With Sophisticated Counterfactual Images

Xiang Li, Ren Togo, Keisuke Maeda, Takahiro OgawaIEEE Open Journal of Signal Processing 2025 · DOI

Abstract

In deep learning, training data, which are mainly from realistic scenarios, often carry certain biases. This causes deep learning models to learn incorrect relationships between features when using these training data. However, because these models have <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">black boxes</i>, these problems cannot be solved effectively. In this paper, we aimed to 1) improve existing processes for generating language-guided counterfactual images and 2) employ counterfactual images to efficiently and directly identify model weaknesses in learning incorrect feature relationships. Furthermore, 3) we combined counterfactual images into datasets to fine-tune the model, thus correcting the model's perception of feature relationships. Through extensive experimentation, we confirmed the improvement in the quality of the generated counterfactual images, which can more effectively enhance the classification ability of various models.
BibTeX
@article{li2025enhancing,
  author = {Xiang Li and Ren Togo and Keisuke Maeda and Takahiro Ogawa and Ren Togo},
  title = {{Enhancing Classification Models With Sophisticated Counterfactual Images}},
  journal = {IEEE Open Journal of Signal Processing},
  year = {2025},
  doi = {10.1109/ojsp.2025.3530843},
}

Loading related content...

Figure