Abstract
Recently, it has been observed that {0,1,-1}-ternary codes which are simply generated from deep features by hard thresholding, tend to outperform {-1,1}-binary codes in image retrieval. To obtain better ternary codes, we for the first time propose to jointly learn the features with the codes by appending a smoothed function to the networks. During training, the function could evolve into a non-smoothed ternary function by a continuation method. The method circumvents the difficulty of directly training discrete functions and reduces the quantization errors of ternary codes. Experiments show that the generated codes indeed could achieve higher retrieval accuracy.
Introduction
In large-scale image retrieval, hashing is a critical technique used to compress high-dimensional image features into short codes for rapid searching. While binary hashing (using +1 and -1) is common, it often suffers from high quantization error because it forces ambiguous features—those with values close to zero—into one of two extreme states.
Ternary hashing addresses this by introducing a third state, "0," to represent these sparse or ambiguous features. However, ternary codes are notoriously difficult to train. Traditional two-step approaches—learning features first and then applying a threshold—are suboptimal because the feature extraction process is unaware of the eventual discretization. The primary challenge in joint optimization is that discrete functions have zero gradients, which prevents standard backpropagation from working effectively.
Methodology: The Continuation Method
To overcome the non-differentiability of discrete ternary functions, this paper proposes a continuation method that bridges the gap between continuous deep features and discrete codes.
The architecture utilizes a standard CNN (AlexNet) followed by a specialized "hash layer." Instead of a hard threshold, features pass through a smoothed ternary function defined as: $$f(x) = \tanh((x/\alpha)^k)$$ where $\alpha$ is a scale parameter and $k$ is an odd integer.
The core innovation is the continuation trick: training begins with a small $k$ (e.g., $k=3$), making the function smooth and differentiable. As training progresses, $k$ is gradually increased (up to 11), "sharpening" the function. This forces the network to adapt its feature representation as the output converges toward a discrete ternary state ({1, -1, or 0}). During inference, this smoothed function is replaced with a hard threshold to produce the final codes.
Experimental Results
The method was evaluated on three standard benchmarks: CIFAR-10, NUS-WIDE, and ImageNet100. The proposed continuation method was compared against state-of-the-art ternary hashing methods, including Deep Polarized Network (DPN) and Ternary Hashing (TH).
As shown in the results table, the continuation method consistently achieved higher Mean Average Precision (mAP) across all datasets and code lengths (16 to 128 bits). For instance, on CIFAR-10 with 128-bit codes, the method reached an mAP of 0.843, surpassing DPN (0.829) and TH (0.825). These gains demonstrate that reducing quantization error during training leads to more discriminative codes without increasing search-phase computational costs.
Key Insights and Discussion
- Joint Optimization: Learning features and hash codes simultaneously is superior to sequential methods. By embedding the hashing function into the loss calculation, the CNN learns to produce features that are inherently easier to ternarize.
- Gradient Flow: The continuation method provides a differentiable "path" for optimization, starting from a simple continuous problem and ending at a complex discrete one, effectively solving the gradient vanishing problem.
- Mathematical Constraints: The choice of an odd power for $k$ is essential to maintain the sign of the input, allowing the model to distinguish between +1 and -1 states.
While successful, future work could explore modern backbones like ResNet or Vision Transformers (ViT) beyond the AlexNet used here. Additionally, incorporating more specialized loss functions, such as triplet or hinge loss, may further refine the quality of the learned ternary representations.
Summary updated