Neural Architecture Search Without Ground Truth Labels

This direction is split into other directions: self-supervised architecture search. Its last version stays readable and citable here.

Version 1 of 139 papers40 conceptsfrozen 2026-10-04

Traditional Neural Architecture Search (NAS) is notoriously "label-hungry." To find the best-performing model, these algorithms typically follow a loop: propose a candidate, train it to completion on a labeled dataset, record its accuracy, and repeat. This process is not only computationally ruinous but also fundamentally impossible in domains where human-annotated labels are scarce or non-existent.

A growing body of research suggests that the "label bottleneck" is an artificial constraint. By shifting the search objective from absolute supervised accuracy to intrinsic structural properties, researchers are discovering that high-quality architectures can be found using self-supervision, theoretical indicators, or even completely random noise.

The Proxy Shift from Labels to Self-Supervision

The core intuition behind label-free NAS is that a "good" architecture is one that learns useful representations, regardless of whether those representations are mapped to specific human categories. If an architecture excels at a difficult self-supervised task, it likely possesses the structural inductive biases necessary for supervised tasks.

Chenxi Liu, Kaiming He, and Ross Girshick famously challenged the necessity of labels in Are Labels Necessary for Neural Architecture Search?, demonstrating that architectures ranked highly under unsupervised objectives maintain their rank when evaluated on ImageNet.

This has led to the rise of Masked Image Modeling (MIM) as a search proxy. In MAE-NAS, Xiangxiang Chu and colleagues use masked reconstruction to drive architecture innovation, avoiding "performance collapse" in differentiable search. Similarly, Caixia Yan and Xiaojun Chang introduced MaskTAS, which searches for optimal Vision Transformers by evaluating how well candidate structures distill features from a pre-trained teacher—all without a single manual label.

Architecture Ranking via Theoretical Indicators

If we can identify a "good" architecture without training it, we eliminate the need for labels entirely. This "training-free" approach relies on mathematical proxies that correlate with a network's potential.

Wuyang Chen and Zhangyang Wang developed TE-NAS, which uses the Neural Tangent Kernel (NTK) and the number of linear regions to rank architectures at initialization. By analyzing the spectrum of the NTK, they can predict how well a network will converge before it ever sees a data point. This allows for architecture discovery on ImageNet in just four GPU hours.

Perhaps most surprisingly, Xuanyang Zhang and Jian Sun 0001 showed in Neural Architecture Search With Random Labels that even random labels can guide the search. By measuring the "weight-angle" metric—how quickly weights change direction during early training—they found that architectures that converge quickly on noise also tend to perform best on real data.

Learning the Predictor with Minimal Supervision

When some labels are available but expensive, researchers use semi-supervised meta-learning to bridge the gap. Instead of training every candidate, they train a Performance Predictor that learns to map an architecture's graph structure to its expected accuracy.

  • Graph-Based Learning: Yehui Tang and Yunhe Wang 0001 proposed a Semi-Supervised Assessor using Graph Convolutional Networks (GCNs) to leverage thousands of "unlabeled" (untrained) architectures.
  • Contrastive Pre-training: Shenghe Zheng and Hongzhi Wang 0001 introduced DCLP, which uses curriculum contrastive learning to pre-train predictors on unlabeled architecture graphs, significantly reducing the number of fully trained models needed for fine-tuning.
  • Data Augmentation: To solve the data scarcity of architecture-performance pairs, Xiangning Xie and Yanan Sun 0001 developed GIAug, which uses graph isomorphism to generate new training samples for predictors "for free."

Bridging the Gap in Unsupervised Domains

The most immediate impact of label-free NAS is in fields where labels are fundamentally missing, such as Unsupervised Domain Adaptation (UDA) and Graph Learning.

In UDA, we have labels for a source domain but none for the target. Luca Robbiano and Fabio Galasso developed ABAS, which uses an ensemble-based estimator to search for optimal adversarial branches that align source and target distributions without target labels. Similarly, AutoAdapt by Xueqing Deng uses regional prediction consistency as a label-free metric to discover segmentation networks.

For graph data, Xin Wang 0019 and Zeyang Zhang 0001 introduced DSGAS, the first framework for unsupervised Graph NAS. It uses disentangled self-supervision to match specific graph motifs (like local clusters) to the neural operations best equipped to process them, proving that even the most complex structural designs can be automated without human intervention.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_neural_architecture_search_without_groun_424201f3,
  title = {Neural Architecture Search Without Ground Truth Labels},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-architecture-search-without-ground-truth-labels/rdr_424201f3895a4f1bb56097c162801f43},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_neural_architecture_search_without_groun_424201f3_v1,
  title = {Neural Architecture Search Without Ground Truth Labels},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-architecture-search-without-ground-truth-labels/rdr_424201f3895a4f1bb56097c162801f43/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}