Introduction
Self-supervised learning (SSL) has revolutionized computer vision by enabling models to learn powerful representations without manual labels. The dominant paradigm, instance discrimination, typically involves teaching a model that two augmented "views" of the same image should have similar representations, while views of different images should be distinct. However, this approach is inherently limited: it treats all other images as negatives, ignoring the fact that many images in a dataset share semantic categories.
To address this, researchers introduced Nearest Neighbour Contrastive Learning (NNCLR), which treats an image's closest neighbors in feature space as positive samples. While this introduces semantic diversity, relying on a single neighbor is often noisy. Conversely, methods that use multiple neighbors (like MSF) suffer from high computational costs. Furthermore, many SSL methods are prone to "feature redundancy," where different dimensions of the embedding vector carry identical information, necessitating extremely large embedding sizes to maintain performance.
The All4One Framework
All4One is a novel SSL framework designed to overcome these limitations through a "symbiotic" architecture. It integrates three distinct objective functions to learn robust, semantically rich, and non-redundant features.
Methodology
The framework utilizes a dual-encoder setup (a standard encoder and a momentum-updated encoder) and maintains a Support Set (a queue of recent representations) to identify neighbors. The three core objectives are:
- Neighbour Contrast (The "Green" Path): Following the NNCLR paradigm, this objective contrasts a representation with its single nearest neighbor from the support set. This ensures the model captures local data density.
- Centroid Contrast (The "Purple" Path): This is the framework's primary innovation. Instead of calculating individual losses for $k$ neighbors, All4One employs a Transformer encoder to aggregate $k$ neighbors into a single "centroid" representation. This centroid acts as a stable, denoised prototype of the semantic neighborhood. The model then contrasts these centroids across different views, gaining the benefits of multi-neighbor methods with significantly lower computational overhead.
- Feature Contrast (The "Red" Path): Inspired by Barlow Twins, this objective pushes the cross-correlation matrix of the representations toward the identity matrix. By minimizing redundancy between different feature dimensions, the model is encouraged to learn independent and informative components.
To maintain a strong gradient signal, the authors implement a Shift Operation, ensuring the original distorted image is always included in the neighborhood sequence processed by the Transformer.
Experimental Results
The model was rigorously evaluated on standard benchmarks, including CIFAR-10, CIFAR-100, and ImageNet-100.
Key Findings
- State-of-the-Art Performance: All4One consistently outperforms its predecessors. On ImageNet-100, it achieved 81.93% Top-1 accuracy, surpassing both NNCLR (79.80%) and Barlow Twins (80.38%).
- Dimensionality Resilience: A significant advantage of All4One is its robustness to low-dimensional embeddings. While redundancy-reduction methods like Barlow Twins typically require large feature vectors, All4One maintains high accuracy even at 256 dimensions, outperforming Barlow Twins by 10% on CIFAR-100 in that configuration.
- Generalization and Robustness: The model demonstrates superior performance in transfer learning tasks (e.g., Food101, Caltech-101) and remains effective even under "weak" data augmentation, suggesting it learns more fundamental semantic structures rather than relying on augmentation-specific cues.
Conclusion and Future Work
All4One demonstrates that combining neighborhood aggregation via self-attention with feature redundancy reduction creates a powerful synergy for self-supervised learning. By distilling neighborhood information into a single centroid, it achieves the performance of multi-neighbor methods with the efficiency of instance-based approaches.
While the framework introduces some complexity through its triple-loss objective and internal Transformer, its resilience and accuracy make it a compelling choice for resource-constrained environments requiring compact embeddings. Future research will focus on applying All4One to larger backbones and dense prediction tasks such as object detection and segmentation.
Summary updated