Most machine learning models operate under a "closed-world" assumption: the set of categories they predict is fixed at the moment the training script finishes. In this paradigm, a label is merely an integer index—a meaningless pointer to a specific output neuron. However, real-world data is rarely so cooperative. New products enter e-commerce catalogs, job markets invent new skills, and scientific consensus refines biological classifications.
When the taxonomy shifts—whether through the addition of new labels, the merging of existing ones, or a complete reorganization of the hierarchy—traditional models break. To fix them, researchers are moving away from treating labels as static identifiers and toward treating them as dynamic, semantic entities.
From Label IDs to Semantic Embeddings
The fundamental intuition behind this research direction is that if a model understands the meaning of a label, it can generalize to new categories without being explicitly retrained on them.
In Cross-Domain Label-Adaptive Stance Detection, Momchil Hardalov and Preslav Nakov introduce MoLE (Mixture-of-Experts with Label Embeddings). Instead of predicting a fixed class index, the model learns to map inputs into a shared semantic space where the labels themselves also reside. By leveraging the textual names of labels, the model can predict a "stance" on a topic it has never seen before, simply by calculating the proximity between the input text and the label's embedding.
This shift from "classification" to "matching" is a recurring theme. Shansan Gong and Xuezhi Cao formalized this in Transferable and Efficient: Unifying Dynamic Multi-Domain Product Categorization with their TaLR framework. By treating categorization as a retrieval task, they allow business taxonomies to split or merge labels instantly. Because the model matches product descriptions to "meta concepts" (shared keywords) rather than rigid IDs, the system remains robust even when the underlying organizational structure is overhauled.
Navigating Shifting Hierarchies
In many domains, labels aren't just flat lists; they are nodes in a deep hierarchy. When these hierarchies evolve, the challenge is maintaining "label consistency"—ensuring that a model doesn't predict a sub-category (e.g., Golden Retriever) without also acknowledging its parent (Dog).
Chao Yu and Yi Shen addressed this in Constrained Sequence-to-Tree Generation for Hierarchical Text Classification. Their Seq2Tree model treats the taxonomy as a path to be generated. By using a constrained decoding strategy, the model can adapt to dynamic environments where categories are added or removed by simply updating the "valid paths" the decoder is allowed to take, rather than retraining the entire network.
Similarly, Tianyi Liu and Hao Wang explored how to handle noise and shifts in these structures. In Taxonomy-Structured Domain Adaptation, they argue that rigid hierarchies can actually introduce incorrect inductive biases if they are slightly inaccurate or outdated. Their solution is a "taxonomist" module that learns to respect the hierarchy while remaining flexible enough to account for relationships that shift over time.
Learning in the Stream
In high-velocity environments like ad-tech or social media, labels don't just change; they emerge in a continuous stream. Zhen Wang and Dacheng Tao proposed Deep Streaming Label Learning (DSLL) in Deep Streaming Label Learning to handle this "infinite" label space.
The intuition here is feature distillation: as new labels appear, the model uses the knowledge stored in historical models to learn the correlations between old and new categories. This prevents the "catastrophic forgetting" typical of neural networks, allowing the system to expand its vocabulary without the prohibitive cost of retraining on the entire historical dataset.
Retrieval as the Ultimate Adaptability
The most recent frontier in this area involves Retrieval-Augmented Generation (RAG). If the taxonomy is stored in an external index rather than the model's weights, updating the model becomes as simple as editing a text file.
Kelvin Guu and researchers at Google demonstrated this with CASPER in Controllable Semantic Parsing via Retrieval Augmentation. By retrieving exemplar parses from an index, developers can rename labels or refactor schemas instantly. This approach is echoed in Graph-based Retrieval Augmented Generation for Dynamic Few-shot Text Classification by Haoyang Li, where a graph-based retrieval mechanism provides Large Language Models with the precise "clues" needed to classify text into rapidly shifting categories.
Summary of Intuition
The transition from static to dynamic taxonomies works by breaking the hard-coded link between a model's architecture and its label set. By representing labels as semantic vectors, hierarchical paths, or retrievable exemplars, these systems treat classification as a flexible matching problem. This allows them to stay relevant in a world where the way we categorize information is constantly in flux.