A voice assistant parses each user utterance into a small structured record. It assigns the sentence an intent, meaning the user's goal, such as BookRestaurant. It also fills slots, meaning labeled spans of words, such as city: "Paris" or party_size: "four". Labeling slots word by word is slow and costly, and every new domain arrives with almost no labels but plenty of unlabeled user logs. This proposal rests on one observation. Almost every such utterance splits into two parts. The carrier phrase ("book a table for ___ in ___") mostly reveals the intent. The value spans ("four", "Paris") are the slots. Those two parts can check each other on unlabeled text.
We propose a semi-supervised method built on that split. The model tags an unlabeled utterance. Its slot predictions are then used to cut the utterance into carrier and values, and the model must agree with itself across those cuts. The intent predicted from the carrier alone must match the intent predicted from the whole sentence. A value span moved into another carrier with the same slot type must be re-tagged as the same slot. Utterances whose predicted parse passes these checks become pseudo-labeled training data. Swapping values between carriers that passed the checks produces new training sentences whose word-level labels are correct by construction. The claim is that these cross-task checks filter pseudo-labels more accurately than model confidence does, and that the boundary-preserving recombination closes the gap where sentence-level augmentation helps intent accuracy but not slot filling.
Where this sits
Joint modeling is well motivated in this literature. Shared encoders, slot-gated attention, heterogeneous graphs and self-distillation loops all show that the intent and slot decisions inform each other (Slot-Gated Modeling for Joint Slot Filling and Intent Prediction; Co-Guiding for Multi-Intent Spoken Language Understanding; Joint Multiple Intent Detection and Slot Filling via Self-distillation). All of these assume full supervision, and Co-Guiding's graph edges and contrastive losses explicitly require gold labels. We reuse their premise that the two tasks constrain each other, but we turn that constraint into a training signal on unlabeled text rather than an extra architectural module.
Work on label-efficient joint parsing takes three routes, and none of them uses unlabeled in-domain logs as a signal tied to both tasks.
The first route is metric learning. Learning to Bridge Metric Spaces (ConProm) merges intent and slot prototypes for few-shot transfer across domains. It reports that generic slots like time blur intent prototypes. Few-shot Intent Classification and Slot Filling with Retrieved Examples retrieves labeled spans. It suffers false positives when a keyword appears in an unrelated context. Both methods use only the labeled support set.
The second route uses augmentation and supervised contrastive losses. Semi-Supervised Few-Shot Intent Classification and Slot Filling found that backtranslation and EDA (simple word-level edits such as synonym swaps) help intent classification but give diminishing returns on slots, because they break span boundaries. Its boundary-safe alternative, swapping values from a slot lexicon, needs a hand-built dictionary. That paper motivates our recombination step. Our step is dictionary-free because both carriers and values come from the model's own validated parses of unlabeled logs.
The third route is active learning. A Multitask Active Learning Framework for Natural Language Understanding computes the exact joint entropy of the intent label and the slot sequence under a CRF (a conditional random field, a sequence model that scores legal tag transitions). Querying by this joint entropy cuts the labels needed by over 40% relative to random sampling. That joint entropy is the natural confidence measure for pseudo-labeling, so it supplies our strongest baseline rather than our method.
The generative route, UGen-DP, prompts T5 with label descriptions. It still needed an explicit intent-to-slot auxiliary task to keep frames consistent, and it stayed unstable on unseen labels. That constrains us: label descriptions alone do not replace structural coupling.
Core Claim
Fix a joint tagger $f$ with an intent head and a CRF slot head, trained on a small labeled set $L$ and applied to a large unlabeled pool $U$. For an unlabeled utterance $u$, let $\hat s(u)$ be the predicted slot spans. The carrier $c(u)$ is $u$ with each predicted span replaced by a placeholder token naming its type, for example [city]. We accept the predicted frame of $u$ only if three checks hold.
- Carrier agreement. The intent predicted from $c(u)$ matches the intent predicted from $u$.
- Compatibility. Every predicted slot type has non-negligible compatibility with the predicted intent. Compatibility comes from a smoothed intent–slot co-occurrence matrix estimated on $L$.
- Re-tagging stability. Each value span, inserted into a different accepted carrier with the same placeholder type, is re-tagged with that type and the same boundaries.
Accepted frames supply pseudo-labels. Accepted carriers and values are recombined into new sentences labeled by construction. Training uses
$$\mathcal{L} = \mathcal{L}_{sup}(L) + \lambda_1 \mathcal{L}_{sup}(A \cup R) + \lambda_2 \sum_{u \in U} \mathrm{KL}\big(p(y \mid u)\,\|\,p(y \mid c(u))\big).$$
Here $A$ is the set of accepted pseudo-labeled utterances, and $R$ is the set of recombined sentences. The term $p(y \mid \cdot)$ is the intent distribution, computed from either the full utterance or the carrier view. The weights $\lambda_1, \lambda_2$ trade off the pseudo-label and consistency terms. The model is retrained, and the cycle of re-parsing, re-filtering and recombining repeats a few times.
The specific new claim is this: at equal acceptance rate, the three cross-task checks yield more accurate pseudo-frames than thresholding on joint entropy. Slot F1 is where most of the gain should appear.
Research Program
We first build the joint tagger: a RoBERTa encoder with an intent head and a CRF slot head. This matches the backbone class used by LCAN and the semi-supervised few-shot work, and the CRF also prevents the illegal tag sequences ConProm reports.
For multi-intent data, the intent head is multi-label. Carrier agreement then compares predicted intent sets, and recombination only swaps values while keeping each carrier whole. Splitting multi-intent carriers at conjunctions is the known hard case (LCAN), so we deliberately avoid it.
The analysis has two parts. The first is a pseudo-label audit. Gold labels exist for the "unlabeled" pool in our simulated settings, so we can measure, for each filter, what fraction of accepted frames are exactly correct as the acceptance rate varies.
The second is a breakdown by slot type. We separate generic slots (time, date, location) from domain-specific ones. ConProm's shared-slot noise predicts that generic slots will often pass the compatibility check and contribute little evidence about intent. Carrier agreement should be immune to this because it removes value spans entirely.
A known risk is that the compatibility matrix estimated from very few labels will be sparse. It could then reject correct frames that contain rare slot types. We will report the rejection rate on gold-correct frames for each slot-frequency bucket, and we will test a version with check 2 removed.
Baselines
The do-nothing reference is the same joint tagger trained only on the labeled fraction $L$, with the pool discarded. The ceiling is that tagger trained on all labels. For context on that ceiling, LCAN reports 59.5% joint accuracy on MixATIS and 85.3% on MixSNIPS with full data (LCAN).
The strongest competitor on our specific question is plain self-training filtered by joint confidence. We will use the same tagger, the same unlabeled pool and the same number of rounds. Pseudo-frames are accepted by thresholding the exact joint entropy of the intent label and the CRF slot sequence, following A Multitask Active Learning Framework for Natural Language Understanding. That paper did not use joint entropy for pseudo-labeling and reports only a label-reduction figure (over 40% fewer labels needed than random), not absolute accuracies. This is the right opponent because it already captures uncertainty across both tasks, so any win must come from the cross-task checks rather than from simply using unlabeled data. For fairness, we tune its threshold so that its acceptance rate matches ours at every round, and we give it the same hyperparameter search budget. Beating it means higher slot F1 and joint accuracy at matched acceptance rate, averaged over at least five label-subset seeds, with non-overlapping confidence intervals.
A second competitor is domain-adaptive continued masked-language-model pretraining on the same unlabeled pool, followed by supervised fine-tuning. This is the simplest way to put the self-supervised pretraining lesson from medical imaging (Advancing Radiograph Representation Learning with Masked Record Modeling) into dialogue text. We also run it in combination with our method, because the two are complementary rather than exclusive.
The augmentation baselines are EDA and backtranslation, plus Slot-List Values substitution as in Semi-Supervised Few-Shot Intent Classification and Slot Filling. Slot-List Values uses a gold-derived lexicon, which makes it an oracle-dictionary reference. Our dictionary-free recombination should approach it.
On MixATIS and MixSNIPS at 10% labels, we compare against UGen-DP. It reports overall-accuracy gains over the UGEN generative baseline of 2.7% on MixATIS and 1.4–2.7% overall at 10% data (UGen-DP). No absolute 10%-split value was found on Lacuna for UGen-DP, so we will rerun it on our splits.
Three ablations isolate the ingredients. The first keeps the agreement filter and removes recombination. The second keeps recombination and removes the filter. The third swaps value spans at random regardless of predicted type, which tests whether gains come from correct labels or merely from extra sentences.
Why not just…?
Why not just self-train with a joint-entropy threshold? This is the real threat. If confident predictions are already correct predictions, cross-task checks add nothing. Our reason to expect a gap is that confidence and correctness diverge exactly where few-shot slot tagging fails. The FewJoint benchmark shows slot F1 far below intent accuracy at 3-shot: 40.37 versus 78.46% for JointProto (FewJoint). A tagger can confidently miss a boundary while the sentence-level intent stays right. The matched-acceptance-rate pseudo-label audit settles this directly. If joint-entropy filtering matches our precision, the filtering claim fails, and we would report the method as recombination only.
Why not just continue masked-language-model pretraining on the logs? Continued pretraining shapes the encoder's representations but never produces word-level labels. The augmentation gap in Semi-Supervised Few-Shot Intent Classification and Slot Filling is specifically about slot supervision. The continued-pretraining baseline and the combined run will show whether its gains are additive with ours or whether they replace them.
Why not just use EDA or backtranslation? The same paper reports that these help intent classification but give diminishing returns on slots, because they disturb span boundaries. Recombination keeps boundaries exact by construction. Slot F1, compared against the EDA and backtranslation runs, tests this.
Why not just prompt a generative model to label the pool? UGen-DP shows that descriptive prompts with T5 remain unstable on unseen labels, and that an explicit intent-to-slot constraint was still needed for consistent frames (UGen-DP). We found no reported result on Lacuna for large-model zero-shot labeling of these benchmarks. Model-generated labels could, however, be fed through our filter, and we include that as an optional arm.
Why not just fine-tune on the few labels at test time? On FewJoint, support-set fine-tuning raises sentence accuracy to 38.97%, but intent accuracy drops from 78.46% to 73.82% (FewJoint). Our do-nothing baseline is exactly this kind of fine-tuning, so any gain over it is measured against that strategy.
Proposed Experiments
The main setting simulates a new domain on ATIS and SNIPS. We keep 1%, 5% and 10% of training utterances as labeled data and treat the rest, stripped of labels, as the pool. Metrics are intent accuracy, slot F1 and joint (sentence) accuracy, over five seeds.
For context, the closest reported few-shot numbers on SNIPS are as follows. The semi-supervised prototypical method reaches 92.1% intent accuracy and 0.619 slot F1 at $K_{max}=20$ (Semi-Supervised Few-Shot Intent Classification and Slot Filling). ConProm reaches 40.20% joint accuracy at 5-shot (Learning to Bridge Metric Spaces). These are episodic protocols, so they are reference points rather than direct comparisons.
The multi-intent setting uses MixATIS and MixSNIPS at the 10% split, with overall accuracy as the headline metric. MixATIS has a mismatch between training and test labels, which also probes rare-label behavior.
As a secondary, transductive check, we use FewJoint's 9 test domains at 3-shot. There the query utterances of each test domain serve as the unlabeled pool. The reference is JointProto+Finetune at 38.97% sentence accuracy (FewJoint). These pools are small, so this tests whether the method helps at all when little unlabeled data is available.
Three outcomes decide the proposal. First, the accepted-frame precision of the cross-task filter must beat joint-entropy filtering at matched acceptance rates on ATIS and SNIPS. Second, the end-task gain over self-training with joint confidence must come mostly from slot F1 rather than intent accuracy. Third, type-matched recombination must clearly beat random swapping and approach the gold-lexicon Slot-List Values reference. If the first check fails, the filtering claim is wrong. If the third fails, the gain does not come from correctly labeled recombined sentences.
References
- Slot-Gated Modeling for Joint Slot Filling and Intent Prediction
- Co-Guiding for Multi-Intent Spoken Language Understanding
- Joint Multiple Intent Detection and Slot Filling Via Self-Distillation
- Learning to Bridge Metric Spaces: Few-shot Joint Learning of Intent Detection and Slot Filling
- Few-shot Intent Classification and Slot Filling with Retrieved Examples
- Semi-Supervised Few-Shot Intent Classification and Slot Filling
- A Multitask Active Learning Framework for Natural Language Understanding
- Promoting Unified Generative Framework with Descriptive Prompts for Joint Multi-Intent Detection and Slot Filling
- LCAN: A Label-Aware Contrastive Attention Network for Multi-Intent Recognition and Slot Filling in Task-Oriented Dialogue Systems
- Advancing Radiograph Representation Learning with Masked Record Modeling
- FewJoint: A Few-shot Learning Benchmark for Joint Language Understanding