Distinguishing Visually Similar Images: Triplet Contrastive Learning Framework for Image-text Retrieval
Lists
Only you see this
Abstract
In recent years, contrastive learning techniques, particularly InfoNCE loss, have propelled advancements in image-text alignment. However, aligning indistinguishable image-text pairs remains a challenge for conventional methods, often overlooking semantically similar content. To overcome these limitations, we introduce the Triplet Contrast Learning Framework (TCLF). Accompanying TCLF is the Stricter Noise Contrast Estimation (SNCE) loss, designed to minimize mutual information between positive and negative image-text pairs. Additionally, we propose the intra-modal mutual information (IMI) loss to encourage a uniform distribution of image features and enhance discriminative capacity for similar images. SNCE aligns two enhanced image versions, and collectively, SNCE and IMI empower TCLF for effective learning with challenging image-text pairs. Experimental evaluations on MSCOCO and Flickr30K datasets demonstrate superior performance compared to state-of-the-art methods. Ablation studies confirm the efficacy of SNCE and IMI in overcoming identified challenges.
BibTeX
@inproceedings{ouyang2024distinguishing,
author = {Pengxiang Ouyang and Jianan Chen and Qing Ma and Zheng Wang and Cong Bai},
title = {{Distinguishing Visually Similar Images: Triplet Contrastive Learning Framework for Image-text Retrieval}},
booktitle = {ICME},
year = {2024},
doi = {10.1109/icme57554.2024.10687694},
}Loading related content...