Label-Free Neural Architecture Search

24 concepts14 papers1 proposal

Traditional Neural Architecture Search (NAS) relies on a simple but expensive feedback loop: generate a candidate network architecture, train it on a massive human-annotated dataset like ImageNet, evaluate its validation accuracy, and use that accuracy as a reward signal to discover better structures.

As noted by Wuyang Chen et al. in Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective, this heavy dependency on labeled data severely restricts our ability to automate model design for unsupervised tasks or data-scarce domains where ground-truth labels are unavailable or prohibitively expensive to obtain.

However, a powerful counterintuitive insight has emerged: human-annotated labels are completely unnecessary for ranking and discovering high-performing neural network architectures. Instead, the structural fitness of a network can be evaluated using self-supervised pretext tasks, training-free theoretical indicators, or even completely random labels.

The Core Intuition: Data Statistics Over Semantic Labels

Why does an architecture discovered without labels perform just as well—and sometimes better—on downstream supervised tasks?

The answer lies in what an architecture search actually optimizes. NAS is not searching for specific semantic features (like "dog" vs. "cat"); it is searching for a structural blueprint capable of efficiently processing the underlying mathematical properties and spatial topologies of natural data.

In Are Labels Necessary for Neural Architecture Search?, Chenxi Liu et al. demonstrated that the relative performance ranking of candidate architectures remains remarkably consistent whether they are trained on supervised tasks or unsupervised pretext tasks, such as predicting image rotation or solving jigsaw puzzles. The internal statistical regularities of natural images provide a rich enough signal to distinguish superior network designs from inferior ones.

To isolate structural capacity from semantic accuracy completely, Xuanyang Zhang et al. pushed this concept to its logical extreme in Neural Architecture Search With Random Labels. By training a super-network using completely fixed random labels, the authors shifted the search metric from "highest accuracy" to "fastest convergence" (measured via a novel weight-angle metric). This approach successfully identified high-performing architectures, proving that a network's innate fitness is encoded in how its structural geometry interacts with data manifolds, entirely independent of label correctness.

Mechanisms of Self-Supervised Evaluation

To replace validation accuracy as a fitness metric, label-free NAS frameworks leverage several self-supervised paradigms to evaluate candidate models:

Contrastive Learning and Consistency

Instead of minimizing cross-entropy against manual labels, models can be evaluated on their ability to map different augmented views of the same image to similar representations. In Towards Self-supervised and Weight-preserving Neural Architecture Search, Zhuowei Li et al. utilized a contrastive learning objective to guide the search process.

Similarly, Sharath Girish et al. demonstrated in Moving Beyond Handcrafted Architectures in Self-Supervised Learning that integrating NAS directly into self-supervised learning (SSL) frameworks allows the simultaneous optimization of network weights and topologies, yielding "self-supervised architectures" that natively maximize contrastive objectives.

Masked Distillation

For Vision Transformers (ViTs), Caixia Yan et al. introduced MaskTAS in Masked Distillation Advances Self-Supervised Transformer Architecture Search. Instead of relying on downstream accuracy, MaskTAS evaluates candidate ViT architectures based on their feature consistency with a pre-trained teacher model using Masked Image Modeling (MIM), allowing the search to scale effectively without manual labels.

Zero-Shot Theoretical Indicators

Rather than training candidate networks at all, the TE-NAS framework by Wuyang Chen et al. measures two training-free theoretical properties at initialization: the linearity of the Neural Tangent Kernel (NTK) and the number of linear regions in the input space. These metrics serve as direct proxies for a network's trainability and expressivity, compressing the search time on ImageNet to a mere four GPU hours without looking at a single label.

The Hidden Advantage: Escaping Label Constraints

Counterintuitively, removing manual labels does not just match supervised NAS performance—it can actually exceed it.

As argued by Zhuowei Li et al., manual labels impose artificial optimization constraints on a network during the search phase. Forcing a network to collapse complex visual data into a few rigid semantic categories can restrict the search algorithm from discovering more flexible structural configurations. By optimizing for self-supervised objectives instead, the search process uncovers a more generic and robust optimization manifold.

This phenomenon explains why label-free architectures generalize so well. The "universal backbones" discovered on large, unlabeled corpora by Chenxi Liu et al. and Caixia Yan et al. demonstrate superior transferability across diverse downstream tasks, such as semantic segmentation and object detection, outperforming architectures discovered via traditional supervised search.

Generalization Across Domains

The core principles of self-supervised architecture discovery extend far beyond standard image classification:

By decoupling the expensive architecture discovery process from human annotations, self-supervised NAS shifts the paradigm of deep learning design: it allows data to dictate the optimal shape of the network through its own intrinsic geometry.

Key Foundational Work

Are Labels Necessary for Neural Architecture Search? (2020) — contributed 5 concept elements to this direction.
Learning the Structure of Deep Convolutional Networks (2015) — contributed 1 concept elements to this direction.
Adapting Neural Architectures Between Domains (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_self_supervised_architecture_search_2004a92d,
  title = {self-supervised architecture search},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/self-supervised-architecture-search/rdr_2004a92d77b24c4291e18c042d2a8ea3},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_self_supervised_architecture_search_2004a92d_v1,
  title = {self-supervised architecture search},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/self-supervised-architecture-search/rdr_2004a92d77b24c4291e18c042d2a8ea3/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake