Most modern machine learning architectures are built on the assumption of order. Recurrent Neural Networks (RNNs) process tokens one by one, and even Transformers, despite their parallel attention mechanism, are typically trained to generate outputs as linear sequences.
However, many real-world problems do not have a natural order. If you are extracting a list of symptoms from a medical report or identifying the ingredients in a recipe, the order in which you list them is arbitrary. Forcing these "sets" into a "sequence" creates a fundamental structural mismatch that hampers model performance.
The Problem of Forced Ordering
When we use a standard sequence-to-sequence (Seq2Seq) model for a task like Named Entity Recognition (NER), we typically linearize the entities into a string. As Jiangnan Li and Bin Liang observe in Set Learning for Generative Information Extraction, this forces the model to treat different permutations of the same correct information as incorrect.
This creates two primary issues:
- The Entity-Order Confounder: Shuai Zhang and Yongliang Shen identify this as a "confounder" in De-Bias for Generative Extraction in Unified NER Task. The model begins to learn spurious dependencies—for example, believing that "Entity B" only appears if "Entity A" was mentioned first—simply because of the arbitrary order used in the training data.
- Loss Inflation: As Zeqi Tan explains in A Sequence-to-Set Network for Nested Named Entity Recognition, if a model predicts all the correct items but in a different order than the ground truth, standard cross-entropy loss will penalize it heavily. This "noisy" training signal makes the optimization process much harder than it needs to be.
Moving to Sequence-to-Set Architectures
To solve this, researchers are shifting toward Sequence-to-Set (Seq2Set) frameworks. Instead of predicting items one by one in a chain, these models aim to predict the entire collection at once.
A key mechanism in this shift is Bipartite Matching Loss. In Joint Entity and Relation Extraction With Set Prediction Networks, Dianbo Sui and colleagues use a Transformer decoder to produce a fixed number of "slots." They then use the Hungarian algorithm to find the best possible one-to-one matching between these predicted slots and the actual ground-truth items. Because the loss is calculated based on this optimal matching rather than a fixed position, the model is never penalized for the order of its predictions.
Parallel Generation and Non-Autoregression
If the order doesn't matter, why generate items one by one? Buzhou Tang and Yuxin He propose SetGNER, which treats entity extraction as a parallel generation task. By detecting entity "heads" and generating their full mentions concurrently, they eliminate the sequential bottleneck entirely.
Similarly, Penghui Wei uses non-autoregressive decoding in Conna to generate "bundles" of items for advertisements. This approach is not only more theoretically sound for unordered data but also significantly faster, as the model doesn't have to wait for the previous token to finish before starting the next.
Handling Cardinality and Diversity
A set isn't just unordered; it also has a variable size (cardinality). Standard Seq2Seq models use an "End of Sentence" (EOS) token to stop, but this is less intuitive for sets.
In One Size Does Not Fit All, Xingdi Yuan and Tong Wang address this by treating keyphrase generation as a set problem, allowing the model to dynamically determine how many phrases to produce based on the source content. To prevent the model from repeating the same item in different slots, Jie Chen and Tian Gao introduce a "memory penalty" in their Sequential Set Generation method, which discourages the model from revisiting parts of the input it has already "covered."
Beyond Text: Sets in Vision and Flow
The realization that "everything is a set" is spreading beyond NLP. Zigang Geng and Han Hu recently proposed Tokenize Image as a Set, arguing that even image representations shouldn't be confined to rigid grids or sequences.
For even more complex data like point clouds, Chaoyu Liu and Carola-Bibiane Schönlieb developed Generative Unordered Flow. This framework treats sets as continuous functions in a Hilbert space, allowing for the generation of unordered data points through flow matching, bypassing the limitations of traditional diffusion models that struggle with permutation invariance.
By aligning the mathematical structure of our models with the unordered nature of the tasks they solve, we move away from "teaching the model to lie" about sequences and toward a more robust, efficient form of machine intelligence.