Introduction
Deep learning models are notoriously data-hungry, yet for many regression tasks, obtaining high-quality labels is a significant bottleneck. In Image Quality Assessment (IQA), a single reliable score often requires the consensus of dozens of human raters. Similarly, in Crowd Counting, researchers must manually annotate every individual in images containing thousands of people. This "labeling tax" limits dataset sizes and hinders model generalization.
This paper introduces a self-supervised framework that bypasses this bottleneck by using "learning to rank" as an auxiliary task. By automatically generating ranked image sets from unlabeled data—leveraging synthetic distortions or geometric properties—the authors enable models to learn the underlying logic of a task without human intervention.
Methodology: Multi-task Siamese Learning
The core of the proposed approach is a multi-task Siamese network designed to learn from both labeled and unlabeled data simultaneously.
The architecture consists of three main components:
- Shared Backbone: A CNN (e.g., VGG16) extracts features from both labeled and ranked image sets.
- Regression Branch: Trained on labeled data to predict absolute values (e.g., quality scores or person counts).
- Ranking Branch: Trained on unlabeled, automatically ranked pairs using a pairwise ranking loss ($L_{rank}$).
To address the computational overhead of Siamese networks, the authors implemented an efficient backpropagation layer. Instead of passing image pairs through the backbone multiple times, each image in a mini-batch is processed once. All possible pairs are then constructed within the loss layer, resulting in a speedup proportional to the mini-batch size.
Self-Supervised Ranking Strategies
The effectiveness of the framework relies on "knowing" the relative rank of images without ground-truth labels. The authors propose two domain-specific strategies:
Image Quality Assessment (IQA)
For IQA, the authors utilize synthetic distortions. While the absolute quality of a natural image is unknown, applying increasing levels of Gaussian blur or JPEG compression creates a deterministic quality hierarchy. The model is trained to recognize that an image with higher distortion intensity has lower quality.
Crowd Counting
In crowd counting, the authors employ a "containment" rule. If one image patch is a sub-crop of another, the larger patch must contain a number of people greater than or equal to the smaller one. This geometric guarantee allows for the generation of infinite training pairs from any unlabeled crowd scene.
Key Results and Insights
The multi-task approach yields significant improvements across multiple benchmarks:
- Performance Gains: On the TID2013 IQA dataset, the Spearman correlation (SROCC) improved from a baseline of 0.612 to 0.806. In crowd counting on the UCF-QNRF dataset, the Mean Absolute Error (MAE) was reduced from 137 to 124.
- Labeling Efficiency: By using the model's ranking uncertainty as a proxy for active learning, the authors reached target accuracy levels using 50-75% fewer labeled images compared to random sampling.
- Feature Robustness: Training on ranking tasks forces the network to learn monotonic, robust features that directly enhance the primary regression performance, proving that relative order is a powerful proxy for absolute value.
Conclusion
This research demonstrates that self-supervised ranking is a universal and effective auxiliary task for regression. By exploiting the "free" information available in the physical or geometric properties of unlabeled data, the framework significantly reduces the reliance on expensive human labels. Future work may extend this logic to other domains like age estimation or depth perception, where natural rankings are inherently present.
Summary updated