Abstract
Image copy detection (ICD) aims to determine whether a query image is an edited copy of any image from a reference set. Currently, there are very limited public benchmarks for ICD, while all overlook a critical challenge in real-world applications, i.e., the distraction from hard negative queries. Specifically, some queries are not edited copies but are inherently similar to some reference images. These hard negative queries are easily false recognized as edited copies, significantly compromising the ICD accuracy. This observation motivates us to build the first ICD benchmark featuring this characteristic. Based on existing ICD datasets, this paper constructs a new dataset by additionally adding 100,000 and 24, 252 hard negative pairs into the training and test set, respectively. Moreover, this paper further reveals a unique difficulty for solving the hard negative problem in ICD, i.e., there is a fundamental conflict between current metric learning and ICD. This conflict is: the metric learning adopts symmetric distance while the edited copy is an asymmetric (unidirectional) process, e.g., a partial crop is close to its holistic reference image and is an edited copy, while the latter cannot be the edited copy of the former (in spite the distance is equally small). This insight results in an Asymmetrical-Similarity Learning (ASL) method, which allows the similarity in two directions (the query ↔ the reference image) to be different from each other. Experimental results show that ASL outperforms state-of-the-art methods by a clear margin, confirming that solving the symmetric-asymmetric conflict is critical for ICD.
Introduction
Presented at AAAI-23, this research identifies a fundamental flaw in current Image Copy Detection (ICD) methodologies. By pinpointing a conflict between standard metric learning and the unidirectional nature of image editing, the authors introduce a more rigorous benchmark and a novel architectural solution to improve real-world detection accuracy.
The Hard Negative Problem
Image Copy Detection (ICD) involves identifying if a query image is an edited version (e.g., cropped, rotated, or color-shifted) of a reference image. However, real-world applications are plagued by "hard negatives"—images that look remarkably similar to a reference but are not actually copies. For instance, two different photographs of the same cockpit taken from slightly different angles can easily trigger false positives in standard systems.
Existing benchmarks often lack these distractors, leading to inflated performance metrics that do not hold up in practice. To address this, the authors developed the Negative Distractor for Edited Copy (NDEC) dataset, which incorporates over 124,000 manually annotated hard negative pairs.
The Symmetric-Asymmetric Conflict
A core insight of this paper is the "Symmetric-Asymmetric Conflict." Standard deep metric learning relies on symmetric distance: if Image A is mathematically close to Image B, then Image B is equally close to Image A. However, the editing process is inherently asymmetric. While a small crop is a copy of the original holistic image, the original image is not a copy of the crop.
Symmetric models struggle to distinguish the source from the copy, making them highly vulnerable to hard negatives that share visual features but lack a parent-child relationship.
Asymmetrical-Similarity Learning (ASL)
To resolve this conflict, the authors propose Asymmetrical-Similarity Learning (ASL). This method breaks symmetry by incorporating feature norm ratios into the similarity calculation:
- Feature Norm Ratios: ASL calculates $R(x_i \to x_j) = |x_i| / |x_j|$. The model is trained such that the source image (containing more information) has a larger feature norm than its edited counterpart.
- Directional Learning: By mapping visual information density to vector magnitude, the model can determine the direction of an edit.
- Dual-Objective Loss: The training combines standard distance-based loss with a norm-ratio loss. This ensures that while similar images remain close in vector space, the model can differentiate them based on the "size" of their data.
Key Results and Impact
The introduction of the NDEC benchmark revealed a significant gap in the field; state-of-the-art methods saw performance drops of 23% to 39% when faced with hard negatives. ASL successfully mitigated this, outperforming existing methods by +5.22% μAP on NDEC and reducing false positives by nearly 75% in ablation studies.
While ASL shows great promise, particularly for cropping, future work remains to address more complex edits that might increase information density, such as AI-based outpainting. Nonetheless, this work establishes that solving the symmetric-asymmetric conflict is a critical step toward practical, robust image copy detection.
Summary updated