Supervised Contrastive Learning for Image-to-Graph Transformers

Anna Banaszak, Alexander H. Berger, Laurin Lux, Suprosanna Shit, Daniel Rueckert, Johannes C. Paetzold

Abstract

Image-to-graph transformers can effectively encode image information in graphs but are typically difficult to train and require large annotated datasets. Contrastive learning can increase data efficiency by enhancing feature representations, but existing methods are not applicable to graph labels because they operate on categorical label spaces. In this work, we propose a method enabling supervised contrastive learning for image-to-graph transformers. We introduce two supervised contrastive loss formulations based on graph similarity between label pairs that we approximate using a graph neural network. Our approach avoids tailored data augmentation techniques and can be easily integrated into existing training pipelines. We perform multiple empirical studies showcasing performance improvements across various metrics.
BibTeX
@inproceedings{banaszak2024supervised,
  author = {Anna Banaszak and Alexander H. Berger and Laurin Lux and Suprosanna Shit and Daniel Rueckert and Johannes C. Paetzold},
  title = {{Supervised Contrastive Learning for Image-to-Graph Transformers}},
  booktitle = {GRAIL @ MICCAI},
  year = {2024},
  doi = {10.1007/978-3-031-83243-7\_1},
}

Loading related content...

Figure