Single-Branch Architectures for Efficient Self-Supervised Learning

This direction is no longer present in the current build. Its last version stays readable and citable here.

Version 1 of 112 papers18 conceptsfrozen 2026-10-04

The dominant paradigm in self-supervised learning (SSL) has long been the "dual-branch" or Siamese network. Methods like SimCLR, MoCo, and BYOL work by taking two different augmented views of the same image and teaching a model to recognize that they represent the same underlying concept. While effective, this approach is computationally expensive: it requires twice the forward passes, massive batch sizes, and complex mechanisms like momentum encoders or memory banks to prevent the model from "collapsing" into a trivial solution.

A growing body of research is challenging this status quo by demonstrating that high-quality representations can be learned using single-branch architectures and single-view training. By treating SSL as a massive classification task or a structured reconstruction problem, these methods slash computational overhead while maintaining, or even exceeding, the performance of their dual-branch counterparts.

The Return of Instance Classification

The simplest way to avoid the complexity of dual-branch networks is to return to a classic idea: parametric classification. In this framing, every single image in a dataset is treated as its own unique category.

Yu Liu and colleagues demonstrated the power of this approach in Train a One-Million-Way Instance Classifier for Unsupervised Visual Representation Learning. By scaling a single-branch classifier to handle one million distinct classes, they showed that a model can learn robust features simply by trying to identify which specific image it is looking at. This removes the need for comparing pairs of images entirely, simplifying the training pipeline significantly.

Building on this, Yun-Hao Cao and Jianxin Wu refined the mathematics of instance discrimination in On Improving the Algorithm-, Model-, and Data- Efficiency of Self-Supervised Learning. By introducing a "SqrtKL" self-distillation loss, they achieved a 71% reduction in training time compared to state-of-the-art dual-branch methods, proving that single-branch models are particularly effective for lightweight architectures and limited data scenarios.

Learning from a Single View

If we only have one view of an image, how does the model know what to learn? One answer is cluster discrimination. Instead of comparing an image to its augmented twin, the model compares an image to a set of globally maintained clusters.

Qi Qian and Rong Jin explored this in Unsupervised Visual Representation Learning by Online Constrained K-Means. Their approach uses a single augmented version of each image and assigns it to a cluster. The model learns by trying to predict these cluster assignments, focusing on global structure rather than local instance-level comparisons. This "single-view" strategy drastically reduces memory usage because the model doesn't need to store or process multiple versions of the same data point.

Another strategy is to use the model's own history as a reference point. In Self-Adaptive Training: Bridging the Supervised and Self-Supervised Learning, Chao Zhang and Hongyang Zhang proposed using an exponential moving average of the model's past predictions as a target. This creates a "memory" of what the image looks like, allowing the model to remain consistent over time without needing a second augmented branch in the current training step.

Masking as a Single-Branch Alternative

Masked modeling—popularized by BERT in NLP and MAE in vision—is inherently a single-branch approach. The model sees a partial version of the data and must predict the missing pieces.

Recent work has extended this to more complex domains:

Why It Works: Efficiency and Stability

The move toward single-branch architectures is driven by a need for efficiency. Dual-branch methods are often "heavy," requiring specialized hardware and long training times to converge. By contrast, single-branch methods like those proposed by Ishan Rajendrakumar Dave in TimeBalance: Temporally-Invariant and Temporally-Distinctive Video Representations for Semi-Supervised Action Recognition focus on extracting multiple signals (like temporal invariance and distinctiveness) from the data itself rather than relying on expensive multi-modal inputs or redundant views.

Ultimately, these papers suggest that the "contrastive" signal provided by a second augmented view is not the only way to prevent representation collapse. Whether through massive instance classification, historical consistency, or masked reconstruction, single-branch architectures provide a leaner, faster, and more mathematically direct path to understanding the visual and auditory world.

Papers in this version

History & citationVersion 1 · 2026-10-04

No longer present in the current build · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_single_branch_architectures_for_efficien_de17fa22,
  title = {Single-Branch Architectures for Efficient Self-Supervised Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/single-branch-architectures-for-efficient-self-supervised-learning/rdr_de17fa2217554f8a81d80727d3717431},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_single_branch_architectures_for_efficien_de17fa22_v1,
  title = {Single-Branch Architectures for Efficient Self-Supervised Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/single-branch-architectures-for-efficient-self-supervised-learning/rdr_de17fa2217554f8a81d80727d3717431/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}