Voice assistants have to do two things with every request: decide what the user wants (the intent, for example "book a flight") and mark which words fill which fields (the slots, for example "Boston" as the departure city). Training one shared network on both jobs at once, called multi-task learning, reliably helps. It has one practical cost: every training sentence needs both kinds of label. Intent labels are cheap because they are one tag per sentence. Slot labels are expensive because every word must be tagged. Real data collections therefore look lopsided: many sentences carry only an intent label and a few carry both. Meanwhile, the people who build these systems already write down a piece of external knowledge for free, the domain schema that says which slot types each intent can take. A weather request takes a city and a date. It never takes a cuisine.
This note proposes using that schema as a soft constraint on the model's outputs, so that it becomes the channel through which each task teaches the other when labels are missing. On a sentence labeled only with its intent, the constraint pushes the slot predictions away from slot types that intent cannot take. On a sentence labeled only with slots, it pushes the intent prediction toward intents that can take those slot types. The constraint has adjustable slack: each schema entry has an allowance that grows when the data consistently contradicts it, so a wrong or incomplete schema bends instead of breaking. The claim is specific and falsifiable. When fully labeled sentences are scarce, schema-derived output constraints should recover more slot and sentence-level accuracy from single-task labeled data than masked joint training, self-training, or a label-compatibility prior estimated from the few fully labeled sentences. The advantage should shrink toward zero as full labels become plentiful.
Where this sits
Two existing systems already put knowledge inside joint intent–slot models, and both are the closest prior work. GISCo builds a graph of how often intent and slot labels co-occur across the training corpus. It runs a graph network over those counts to shape label embeddings, reporting a 4.7% absolute gain in overall accuracy on MixATIS over the previous best model. Slot-Gated Modeling learns a gate that lets the sentence-level intent vector condition each word's slot prediction. Both motivate this proposal, because explicit cross-task structure clearly helps. Both also leave its target untouched. GISCo's graph is estimated from sentences that carry both labels and is used as an internal representation, not as a constraint on what the model may output. The slot gate is learned purely from data and runs in one direction only, from intent to slots. The proposal differs from both in three ways: the knowledge comes from the schema rather than from paired counts, it acts on predictions rather than on embeddings, and it runs in both directions.
The mechanism is reused from posterior regularization for cross-lingual sequence models. That work imposes constraints on the model's expected label counts instead of treating noisy projected labels as ground truth, and it beat the treat-as-ground-truth baseline by 4.6 F1 on named-entity segmentation. It applied the idea within a single task, however, and chose the constraint strength with a hand-set heuristic based on the tag-per-token ratio. Here the same idea becomes a cross-task constraint, and constraint strength is learned per schema entry.
Two vision papers show that output-level cross-task consistency is powerful. Cross-task consistency requires that predicting depth directly agree with predicting depth by way of other predicted quantities. Joint-Task Regularization learns an autoencoder over the stacked predictions for all tasks and pulls predictions toward it when labels are partial, reporting +21.92% relative improvement on NYUv2 with random partial labels. These papers constrain the design rather than supply it. Both learn the consistency relation from paired dense data. The first explicitly leaves discrete, word-level tasks open, and neither uses external symbolic knowledge. SELF shows theoretically that known output constraints make tasks share statistical strength. It needs those constraints as continuous equations and solves an optimization at test time. A schema is exactly a known constraint, but a discrete one. The proposal is essentially the discrete, trainable counterpart of SELF's softened-constraint variant.
The soft-versus-hard evidence from knowledge integration sets a firm design rule. Dictionary features used as weighted evidence (Japanese morphological analysis, gazetteer-enhanced NER) consistently beat hard lookups and naive injection of dictionary entries as training data. The constraint must therefore be soft, and it must be able to learn that some entries are wrong.
Core mechanism
Let $x$ be an utterance, $y^I$ its intent, and $y^S$ its slot sequence. The schema is a binary matrix $K$ whose entry $K_{is}=1$ means slot type $s$ is allowed with intent $i$. For each disallowed pair, define a feature $\phi_{is}(x, y^I, y^S)$ that equals one when the intent is $i$ and counts the words tagged with slot type $s$. Following posterior regularization, the model's joint prediction $p_\theta$ (with network parameters $\theta$) is projected onto a nearby distribution $q$ that nearly satisfies the schema:
$$ \min_{q}\ \mathrm{KL}\big(q \,\|\, p_\theta(y^I, y^S \mid x)\big) \quad \text{s.t.}\quad \mathbb{E}_{q}[\phi_{is}] \le \xi_{is}\ \ \text{for all } (i,s) \text{ with } K_{is}=0. $$
Here $\xi_{is}\ge 0$ is a per-entry slack, and any observed label clamps the corresponding variable in $q$. Training alternates two steps. The first computes $q$, which is cheap because slot features factor over words given the intent, so projecting within each intent is a per-word reweighting. The second trains $p_\theta$ toward $q$ on every sentence, together with ordinary supervised loss on whichever labels exist. The slacks are learned with an $\ell_1$ price $\beta\sum \xi_{is}$. A schema entry the data keeps violating therefore grows its allowance, which is the adaptive trust that hand-set constraint strengths lack. For multi-intent data, the constraint applies to the union of predicted intents.
On a sentence labeled only with its intent, the projection moves slot probability mass off incompatible slot types. That gives the slot tagger signal from a sentence with no slot labels. On a sentence labeled only with slots, it moves intent mass toward intents compatible with the observed slot types. This two-way transfer is what the slot gate and GISCo's graph do not provide.
Research Program
The work has three parts. The first builds the method on the public SlotGated-SLU codebase, with its BiLSTM encoder, and on a pretrained-transformer encoder so the result does not hinge on one backbone. The second constructs schemas: from the SNIPS domain definitions, from the ATIS frame definitions, and for MixATIS and MixSNIPS by taking the union over each component intent.
The third creates partially labeled versions of each benchmark. Their training sets are split so that a fraction $\rho$ of sentences keep both labels, with $\rho$ in {1%, 5%, 10%, 30%, 100%}. The remainder keep only intent labels in the realistic regime, or a mix of intent-only and slot-only labels in the symmetric regime.
The analysis asks two things. The first is whether the gain tracks the amount of information the schema carries. SNIPS slot types are nearly disjoint across intents, so the schema is very informative there. ATIS intents share many slot types, so the schema is weaker there. The second is whether the learned slacks correctly flag deliberately corrupted schema entries.
Baselines
The naive reference point is the Slot-Gated joint model trained only on the fully labeled fraction $\rho$. The second reference point is the same model trained on all data with the loss for each missing label masked out. Masked training is what most practitioners would do, and it is the real null hypothesis: shared encoders might already pass enough signal across tasks without any constraint.
The strongest existing method for single-intent data is Slot-Gated Modeling. Under full labels it reports slot F1 of 95.2% and intent accuracy of 94.1% on ATIS, and slot F1 of 88.8% and intent accuracy of 97.0% on Snips. Our reproduction at $\rho=100\%$ must land near these numbers before any partial-label comparison is trusted.
For multi-intent data the strongest method is GISCo. The file reports only its relative gain, +4.7% absolute overall accuracy over GL-GIN on MixATIS and +0.5% on MixSNIPS. No absolute overall-accuracy value and no low-resource number was found on Lacuna, so GISCo will be rerun under our splits. In each partial-label split, its co-occurrence graph will be estimated from the fully labeled sentences, which is the only data from which it can be built.
The key ablations isolate the two ingredients. The first keeps the projection but replaces the schema with a compatibility matrix obtained by thresholding co-occurrence counts from the fully labeled fraction. This tests whether external knowledge matters or just any compatibility matrix does. The second fixes all slacks to a single tuned constant. This tests whether adaptive trust matters.
Every method gets the same encoder, the same data split, the same number of training epochs, and the same budget of twenty hyperparameter trials selected on a dev set with the same partial labeling. Results are averaged over five seeds and five random label splits. Beating the baselines means a sentence-level (overall) accuracy gain over the better of masked training and GISCo that exceeds seed-to-split variation at $\rho \le 10\%$ on at least three of the four benchmarks, with no loss at $\rho=100\%$.
Why not just…?
Why not just mask impossible slots at decoding time? A hard mask uses the same schema and costs nothing. Evidence from soft integration says it fails when knowledge is imperfect. Treating projections as hard observations lost 4.6 F1 to posterior regularization in the cross-lingual study, and hard dictionary lookups broke on unknown words in Japanese morphological analysis. A hard mask applied only at decoding also gives the model no training signal from intent-only sentences. We include hard masking both at decoding and during training, and we also include the schema-corruption test. If the schema is clean and masking matches our method, the learning benefit rests entirely on the training-time projection, and we will report that.
Why not just self-train, filling missing slot labels with the model's own confident predictions? This is the most serious threat, because it also turns intent-only data into slot supervision. Its weakness is that it reinforces the model's own errors. That weakness is sharpest exactly when paired data is scarcest, which is where the schema adds information the model does not have. We run confidence-thresholded self-training with the same tuning budget, plus a combined variant that self-trains on schema-projected predictions. If plain self-training matches our method at low $\rho$, the claim that the knowledge matters is refuted.
Why not just learn the compatibility from the fully labeled data, as GISCo does? At $\rho=30\%$ this estimate may well be good enough, and we expect the schema's advantage to disappear there. The co-occurrence ablation above measures exactly where, as a function of $\rho$, estimated compatibility catches up with written-down compatibility. That crossover point is itself a useful result for practitioners deciding whether to write schemas.
Why not just pretrain on the plentiful intent labels and then fine-tune on slots? Sequential training tends to erode shared representations (Wang et al.). Joint training with balanced sampling avoids early overfitting to the source task (Yang et al.). We nonetheless include this pipeline as a baseline, since it is what many teams would try first.
Proposed Experiments
The primary experiment sweeps $\rho$ on ATIS, Snips, MixATIS and MixSNIPS. It reports slot F1, intent accuracy and sentence-level accuracy (whether the full semantic frame is exactly right) for every method above.
The first decisive check is the shape of the curve. The gain over masked training should be largest at $\rho=1$–$5\%$ and near zero at $\rho=100\%$. It should also be larger on Snips than on ATIS, because the Snips schema is more informative. A flat or reversed pattern would mean the method helps for reasons unrelated to the schema.
The second check corrupts the schema. We randomly remove 10–30% of allowed pairs and add spurious ones, then compare learned slacks, fixed slack and hard masking. The prediction is that hard masking degrades sharply, fixed slack degrades moderately, and learned slack stays near its clean-schema accuracy while assigning its largest allowances to the removed pairs. Recall of corrupted entries among the top-allowance pairs is measured directly.
The third check follows cross-task consistency, whose unlabeled disagreement score correlated with error at $r=0.67$. We test whether the schema violation mass of $p_\theta$ on test sentences correlates with frame error. If it does, the schema also provides a label-free warning signal when the assistant meets unfamiliar requests.
References
- Enhancing Joint Multiple Intent Detection and Slot Filling with Global Intent-Slot Co-occurrence
- Slot-Gated Modeling for Joint Slot Filling and Intent Prediction
- Cross-Lingual Discriminative Learning of Sequence Models with Posterior Regularization
- Robust Learning Through Cross-Task Consistency
- Joint-Task Regularization for Partially Labeled Multi-Task Learning
- Consistent Multitask Learning with Nonlinear Output Relations
- The Unknown Word Problem: a Morphological Analysis of Japanese Using Maximum Entropy Aided by a Dictionary
- Towards Improving Neural Named Entity Recognition with Gazetteers
- Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling
- Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks