Abstract
Interactive systems are promising for content-based image retrieval (CBIR), allowing users to iteratively refine search results. Conventional methods often rely on shared semantic knowledge, such as visual attributes, which introduces a "semantic gap" and requires labeled databases. We present Syntharch, a novel approach that elicits feedback exclusively through AI-synthesized images. By generating options that differ along isolated attribute dimensions, Syntharch eliminates the semantic gap and reduces user confusion caused by attribute correlation in real images. Experimental results on the UT-Zap50K dataset show that Syntharch achieves superior accuracy and consistency compared to retrieval-based baselines.
Introduction
Content-Based Image Retrieval (CBIR) involves locating a specific target image within a massive database. A primary obstacle is the "semantic gap"—the discrepancy between high-level human descriptors (e.g., "elegant" or "sporty") and low-level computational features. Interactive search systems attempt to bridge this gap through iterative feedback. However, traditional text-based queries are often ambiguous. Furthermore, using real images from the database for feedback is problematic; because attributes are naturally correlated (e.g., a "pointier" shoe might also be a different color), users may provide "noisy" feedback that hinders the search algorithm.
Methodology: Synthesis-Driven Search
Syntharch addresses these challenges by utilizing attribute-conditioned synthesis for visual-only feedback. The system employs a RankCGAN generative backbone to learn a multidimensional latent attribute space.
Attribute-Conditioned Synthesis
Instead of retrieving real images, Syntharch synthesizes feedback options that vary along exactly one latent dimension. This allows the system to isolate specific visual properties, ensuring that the user's choice provides "clean" information about the target's location in the attribute space.
Range-Based Interaction and Re-ranking
The search process functions as a multidimensional binary search. In each iteration, the system synthesizes two images representing the 1/4 and 3/4 marks of the current search range for a specific attribute. Upon user selection, the system narrows the search window and employs a probabilistic model to re-rank the entire database, bringing the most relevant images to the top of the results.
Experimental Evaluation
The system was evaluated using the UT-Zap50K dataset, comprising 50,025 shoe images. The model was trained on four relative attributes: open, pointy, sporty, and comfort. A user study involving 10 participants across 300 search sessions compared Syntharch against two baselines: a retrieval-based method using real images ("Retrieved+Range") and an older synthesis method ("Synthesis+Pivot").
Results
Syntharch demonstrated superior performance and greater consistency than the baselines.
As shown in the performance graph, Syntharch achieved an average target rank percentile of 72.36% after 12 iterations, outperforming the "Retrieved+Range" baseline (70.89%) and significantly beating the "Synthesis+Pivot" method (57.13%). Notably, while retrieval-based methods often plateau or dip when the database lacks diverse feedback options, Syntharch's ability to generate precise variations leads to a consistent improvement in accuracy.
Conclusion
Syntharch effectively bridges the semantic gap by replacing ambiguous text with precise, AI-synthesized visual feedback. By isolating visual attributes through synthesis, the system reduces user confusion and improves search efficiency. Future work will focus on improving image realism in sparse regions of the attribute space and exploring unsupervised learning of visual attributes to generalize the method to unlabeled databases.
Summary updated