Few-shot classifiers improve a great deal when they look at all the unlabeled test images together instead of one by one. This is called transductive inference: the model uses the shape of the whole unlabeled query set to correct its class estimates. On standard benchmarks it adds several points of accuracy. The catch is that it needs the whole query batch in hand. Many real deployments do not have that batch. Images arrive one at a time, a decision is needed now, and the mix of classes shifts over time. In that setting, the methods that lead the leaderboards either cannot run or must wait for future data.
We propose to make transduction work on a stream. The idea is to treat the useful part of transduction as a few running statistics rather than a batch computation. These are the offset between support and query features, the class frequencies in the query stream, and the corrected class centers. All three can be updated in constant memory after each new query. Before any real queries arrive, the statistics are seeded with cheap synthetic pseudo-queries sampled in feature space. The number of parameters the model is allowed to adapt grows with how much unlabeled evidence has accumulated. Put in two sentences: transductive gains come from a handful of statistics that can be maintained recursively and bootstrapped with almost-free synthetic features. The right amount of test-time adaptation is not fixed but should grow along the stream.
Core Claim
We claim that a streaming learner can recover most of the batch transductive gain over inductive few-shot classification, and hold that gain under unknown and drifting class imbalance, if it has three ingredients. It maintains recursive estimates of the support–query shift and of the query class distribution. It starts those estimates from a prior built from synthetic features, weighted as a fixed number of pseudo-counts. It schedules adaptation capacity by effective sample size, starting with a few scalars and moving to prototypes and then a low-rank feature adapter. The prediction we will test is specific. A single fixed capacity will be dominated at some point along the stream, either too rigid late or too unstable early. The best fixed capacity should increase with stream position, a within-stream version of the complexity crossover that Neural network language models for low resource languages found when training data grows. That paper motivates our capacity schedule; it does not supply the method.
Concretely, let $f(x)$ be a frozen backbone feature and $\bar{s}$ the mean support feature. After query $x_t$ arrives, we update the query-mean estimate and the class-frequency estimate:
$$\hat{\mu}_t = \hat{\mu}_{t-1} + \tfrac{1}{n_0 + t}\big(f(x_t) - \hat{\mu}_{t-1}\big), \qquad \hat{\pi}_t = (1-\rho)\,\hat{\pi}_{t-1} + \rho\, p(\cdot \mid x_t).$$
Here $n_0$ is the pseudo-count carried by the synthetic prior and $\rho$ is a forgetting rate that lets $\hat{\pi}$ follow drifting class frequencies. Query $x_t$ is classified by shifting its feature by $\hat{\xi}_t = \bar{s} - \hat{\mu}_t$. It is then scored against prototypes by cosine similarity, with the scores reweighted by $\hat{\pi}_t$ instead of a uniform prior. Prototypes are updated with soft pseudo-labels only when their confidence passes a threshold.
What Prior Work Does and Does Not Cover
Three batch methods motivate the statistics we track. Prototype Rectification for Few-Shot Learning (BD-CSPN) computes a shift between support and query means and pseudo-labels queries to pull prototypes toward the true class centers. Its authors name the streaming case as open. A Baseline for Few-Shot Image Classification fine-tunes the network while minimizing prediction entropy on the query set. That gains 2–7 points but runs about 300 times slower than feed-forward methods, and the authors note that it fails when images arrive singly. Language-Aware Information Maximization for Transductive Few-Shot CLIP (LIMO) adds a marginal-entropy term that needs a batch to prevent every query being assigned to one class. We keep the first paper's shift-and-rectify logic and drop the need for a batch. We replace the second's whole-network update and the third's marginal term with a tracked class distribution.
One paper constrains the design. Realistic evaluation of transductive few-shot learning shows that methods assuming balanced query classes, such as PT-MAP, lose 18–20 points under Dirichlet-sampled imbalance and can fall below inductive baselines. The same paper proposes α-TIM, a variant that replaces the standard entropy term with a Tsallis entropy that does not force equal class frequencies. Because of this result, no uniform-marginal term appears anywhere in our method, and the class distribution is estimated rather than assumed. α-TIM itself still computes its entropy term over a batch.
Another paper supplies the online template. Online Positive and Unlabeled Learning turns a batch risk estimator into online gradient updates with logarithmic regret. It uses a linear model, however, and assumes the class prior is known. We borrow its streaming structure but estimate a non-stationary multi-class prior and work on deep features.
Finally, the synthetic-data literature fixes our cost constraint. Generate, Transduct, Adapt: Iterative Transduction with VLMs (GTA-CLIP) calls large language models inside a transductive loop. It has hallucination risk and needs the full test set. Promptagator: Few-shot Dense Retrieval From 8 Examples and TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models show that large-model synthesis works but costs heavy upfront compute. Our synthetic component therefore works in feature space. It samples from Gaussians centered at the support prototypes, using covariances borrowed from the base classes and shrunk toward a simpler estimate. For CLIP, it uses class-prompt text embeddings. Generating this prior takes microseconds, not API calls. Its role is deliberately narrow: it is a prior worth $n_0$ observations that real queries override. It is not treated as training data.
Research Program
We will build the streaming learner on frozen backbones: the ResNet and WRN backbones used in the few-shot papers above, plus CLIP ViT-B/16. The learner will have three capacity levels. Level zero adapts only $\hat{\xi}$ and $\hat{\pi}$, roughly $d + K$ numbers for feature dimension $d$ and $K$ classes. Level one also updates the $K$ prototypes. Level two adds a rank-$r$ linear adapter on the features, trained online with a confidence-weighted cross-entropy on pseudo-labels plus a penalty that keeps it close to identity. The scheduler moves up a level when the effective number of confidently pseudo-labeled queries per class passes a threshold. That threshold is tuned on base-class validation episodes only.
The analysis has three parts. First, we map accuracy against stream position for each fixed level and locate where the curves cross. Second, we fit $n_0$ and $\rho$ on validation streams and study how sensitive results are to each. Third, we derive a simple variance argument for the cold start. The shift-estimate error decays like $1/(n_0 + t)$ times the feature variance, which is standard behavior for a running mean. This predicts that the synthetic prior helps most for the first roughly $n_0$ queries and that its benefit vanishes afterward, and we will check that prediction.
Baselines
The naive reference point is inductive nearest-prototype classification with cosine similarity on the same frozen features, with no use of unlabeled queries. On CUB 1-shot under the imbalanced protocol, Realistic evaluation of transductive few-shot learning reports 70.6% for its inductive baseline. That number marks the floor any streaming method must clear.
The strongest existing method on this question is α-TIM, because it is the only method in the file built for unknown class imbalance. That paper reports 75.7% 1-shot accuracy on CUB under imbalance, against 74.8% for standard TIM. We treat α-TIM run on the full batch as an upper reference that is not realistic in a stream. We also run it in a streaming form: at each step it is rerun on a sliding buffer of the most recent $B$ queries, with $B$ matched so that its per-query compute is at least ours. BD-CSPN gets the same buffered treatment. Its batch results, 70.31% and 81.89% on miniImageNet 1- and 5-shot and 78.74% 1-shot on tieredImageNet, were measured on balanced queries and are not comparable to the imbalanced protocol. We cite them only to confirm our reimplementation on the balanced setting before switching protocols.
The high-capacity competitor is online entropy-minimization fine-tuning in the style of A Baseline for Few-Shot Image Classification, with one gradient step per query. For its batch transductive fine-tuning the file gives 65.73% on miniImageNet 1-shot. For the CLIP arm, the reference is batch LIMO, which the file credits with up to 5 points over prior methods at 4 shots but no per-dataset numbers.
All methods share backbones, episodes, and stream orderings. Each gets the same budget of 50 validation configurations tuned on base-class episodes. For streaming protocols there is no reported result found on Lacuna, so all streaming numbers will be produced by us. Beating the strongest baseline means two things at matched or lower per-query compute. The streaming learner must exceed buffered α-TIM on cumulative stream accuracy averaged over the imbalanced and drifting protocols. And it must close at least half of the gap between the inductive baseline and batch α-TIM.
Ablations isolate each ingredient. Setting $n_0 = 0$ removes the synthetic prior. Replacing $\hat{\pi}$ with a uniform prior removes the class-distribution tracking. Fixing capacity at each single level removes the schedule.
Why not just…?
Why not just rerun a batch transductive method on a sliding buffer? This is the most serious threat. Once the buffer is full and class frequencies are stable, buffered α-TIM may match us. Its expected weak points are three: the first $B$ queries, abrupt changes in class frequency (a stale buffer mixes old and new mixtures), and compute that grows with $B$ at every step. We will report accuracy as a function of stream position and around drift points, not only an average. If buffered α-TIM matches us everywhere at equal compute, the contribution shrinks to a cost reduction, and we will say so.
Why not just stay inductive, given strong pretraining? Dhillon et al. report 89% 5-way 1-shot accuracy from ImageNet-21k pretraining, so inductive may be enough on easy domains. Yet the same work, and the CLIP results of LIMO and GTA-CLIP, find transductive gains of 2–9.5 points persisting over strong backbones. On imbalanced streams, the risk is reversed: Veilleux et al. show that badly designed transduction does worse than inductive. The inductive baseline therefore appears in every plot. If it is never beaten on some dataset, we treat that dataset as evidence that the method is not needed there.
Why not just use richer synthetic data, such as LLM-generated attributes or generated images? GTA-CLIP shows that language-generated attributes help CLIP by about 4 points over TransCLIP, but it pays an upfront LLM cost, needs the full test set, and carries hallucination risk. In our design, synthesis only sets a prior whose influence fades as $1/(n_0+t)$. Richer synthesis could matter only in the first few queries. On the CLIP arm we will add GTA-style attributes to the text prior and measure the gain in early-stream accuracy per dollar of generation cost.
Why not just fine-tune the whole network online? The complexity literature, for example State Encoders in Reinforcement Learning for Recommendation: A Reproducibility Study and Temporal Convolution Networks for Real-Time Abdominal Fetal Aorta Analysis with Ultrasound, finds that high-capacity models overfit small samples. Single-query updates are an extreme small sample. The online fine-tuning baseline tests this directly: we expect it to drift early and possibly recover late. That pattern is exactly what the capacity schedule is meant to exploit.
Proposed Experiments
We build streaming versions of miniImageNet, tieredImageNet, and CUB-200-2011 at 1 and 5 shots, 5-way. Each has three query regimes: balanced, Dirichlet-imbalanced following Veilleux et al., and drifting, where the Dirichlet draw is resampled at random switch points. Streams have length 75 and 300, and each result averages over several thousand episodes with confidence intervals. The CLIP arm uses a subset of the vision-language suite (CUB, Flowers 102, EuroSAT, DTD, Food101) with class-prompt priors. Metrics are cumulative and position-resolved accuracy, accuracy in a window after each drift point, per-query latency, and memory.
Three checks decide the question. First, the fixed-capacity sweep must show crossing curves, with the best level increasing with $t$. If one level dominates throughout, the scheduling claim fails. Second, the $n_0 = 0$ ablation must lose accuracy mainly in the first roughly $n_0$ queries and match the full method thereafter. A gain that persists late would mean the synthetic prior is leaking bias, not acting as a prior. Third, under drift, removing $\hat{\pi}$ tracking should reproduce at least part of the collapse Veilleux et al. found for uniform-prior methods, while the full method stays above the 70.6% inductive floor on CUB 1-shot.
References
- Neural network language models for low resource languages
- Prototype Rectification for Few-Shot Learning
- A Baseline for Few-Shot Image Classification
- Language-Aware Information Maximization for Transductive Few-Shot CLIP
- Realistic evaluation of transductive few-shot learning
- Online Positive and Unlabeled Learning
- Generate, Transduct, Adapt: Iterative Transduction with VLMs
- Promptagator: Few-shot Dense Retrieval From 8 Examples
- TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models
- State Encoders in Reinforcement Learning for Recommendation: A Reproducibility Study
- Temporal Convolution Networks for Real-Time Abdominal Fetal Aorta Analysis with Ultrasound