In traditional deep learning, the final layer of a model is often its most rigid component. Whether it is a softmax layer for image classification or a CRF for sequence labeling, the architecture is typically hard-coded to a specific, predefined set of labels. This "fixed-head" design creates a fundamental bottleneck: to add a single new category, a researcher must often redesign the output layer and retrain the entire system.
A growing body of research seeks to break this rigidity by decoupling the model’s internal representations from its output taxonomy. By reframing classification as a matching, retrieval, or generative task, these researchers are building "universal" models that can adapt to new labels, granularities, and domains without architectural changes.
The Softmax Bottleneck
The standard approach to classification treats labels as indices. If a model is trained to recognize 1,000 classes, the final layer has exactly 1,000 neurons. As Benjamin J. Meyer and colleagues point out in Nearest Neighbour Radial Basis Function Solvers for Deep Neural Networks, this forces the model to learn axis-aligned decision boundaries that are difficult to scale or modify.
This rigidity is particularly problematic in specialized fields like medicine. Quan Liu and the team behind PrPSeg: Universal Proposition Learning for Panoramic Renal Pathology Segmentation note that conventional segmentation models require a complete redesign whenever a new tissue type is introduced. Similarly, in Natural Language Processing, Pratyay Banerjee argues in Knowledge Guided Named Entity Recognition that fixed output layers make models inflexible to the evolving schemas of biomedical text.
Classification as Matching and Retrieval
To solve this, researchers are replacing fixed weights with dynamic queries. Instead of asking "Which of these 10 indices does this input belong to?", the model asks "How similar is this input to the description of Class X?"
One powerful implementation of this is Language-Driven Segmentation. In Language-driven Semantic Segmentation, Boyi Li and Kilian Q Weinberger map image pixels and text labels into a shared embedding space. Because the "labels" are just vectors in this space, the model can segment categories it never saw during training simply by looking up the vector for a new word like "hair" or "hat."
This shift from "indexing" to "matching" also appears in medical imaging. Gao Yu and collaborators introduced VOILA, which replaces traditional classification heads with voxel-to-text contrastive learning. By treating segmentation as a retrieval problem, the model becomes "universal," capable of scaling to any anatomical category described in text.
Reformulating Tasks as Question Answering
Another way to decouple labels is to change the format of the task itself. If a model can answer questions, it can perform classification by "reading" the labels.
In Template-Based Named Entity Recognition Using BART, Leyang Cui and Yu Wu treat NER as a ranking problem. Instead of tagging tokens, they use a language model to score templates like "London is a location entity." This allows the model to transfer knowledge between datasets with entirely different taxonomies.
A similar intuition drives Event Extraction as Multi-turn Question Answering. Fayuan Li and his team found that by asking "Who is the victim?" rather than classifying a token as a VICTIM label, the model generalizes much better to novel event types. This "instruction-following" approach is unified in the X-Shot framework by Wenpeng Yin and colleagues, which reformulates multi-class classification into a binary task: "Does this input match this label description?"
Label Conditioning and Atomic Criteria
If we stop treating labels as monolithic IDs and start treating them as collections of properties, models become even more flexible. Junfei Xiao argues in A Semantic Space is Worth 256 Language Descriptions that traditional "one-hot" approaches ignore the shared traits between objects, like "fur" or "metal."
By conditioning the model on these properties, we can achieve Label Conditioned Segmentation (LCS). As demonstrated by Tianyu Ma in Label conditioned segmentation, a model can use a single-channel output that changes its meaning based on an input "atlas" label. This allows a single, small model to handle a massive number of classes—and even identify sub-structures it wasn't explicitly trained to recognize.
Why It Works: Moving from Islands to Manifolds
The intuition behind this research direction is a shift in geometry. Standard classification treats every label as an isolated island. Decoupled architectures treat labels as coordinates on a continuous manifold of knowledge.
When Sam Wiseman and Karl Stratos proposed Label-Agnostic Sequence Labeling by Copying Nearest Neighbors, they proved that a model doesn't need to "know" what a label is to assign it. It only needs to know that the current context is similar to a context in the training data that was assigned that label.
By moving away from fixed output heads, the field is moving toward models that are truly task-agnostic. Whether it is PUNet using prompts for medical images or GIT using a generative transformer for vision, the goal is the same: an AI that can learn the "how" of a task once, and then apply it to any "what" we describe to it.