TL;DR
Diffusion classifiers excel at zero-shot classification but suffer from a severe bias toward the "majority" data distributions they saw during pretraining. This paper introduces MiPO (Minority Preference Optimization), a self-improving framework that fine-tunes a diffusion model using online reinforcement learning and a novel minority-seeking reward function. Crucially, MiPO requires only text prompts—no additional images or external reward models—to expand the model's perceptual coverage and significantly boost zero-shot classification accuracy on underrepresented data.
What Problem Does This Paper Address?
"Diffusion Classifiers" are a clever twist on generative models: instead of generating images, they use a pretrained text-to-image model's noise-prediction error as a proxy to perform zero-shot classification. While robust against out-of-distribution data, they inherit a fundamental flaw from their generative roots: they are heavily biased toward high-density, "majority" regions of the data manifold.
For instance, if a standard diffusion model is excellent at generating typical "soap dispensers," a diffusion classifier will easily recognize them. However, it will likely misclassify underrepresented, low-density variations (like a soap dispenser shaped like a duck or a lotion bottle) because the underlying model lacks a strong internal representation of these "minority" concepts.
Prior works attempted to fix this by steering generation toward minority samples at inference time, but these methods are computationally expensive and don't update the model's internal representations. This paper answers a key question: Can we fundamentally improve a diffusion classifier's perception by training it to better explore minority data regions?
Methods
The authors propose Self-Improving Diffusion Classifiers with Minority Preference Optimization (MiPO). The core idea is to let the model self-train using text prompts to find and reinforce its own weak spots.
The pipeline consists of four main components:
- Minority Preference Reward: To quantify how "underrepresented" a sample is without an external reward model, MiPO adds noise to a generated image $x_0$ at an intermediate step ($t=0.9T$) and tries to reconstruct it ($\hat{x}_0$) using a DDIM-based approximation. Images with higher reconstruction errors ($\|x_0 - \hat{x}_0\|_2$) signify regions where the denoiser is highly uncertain—meaning it's a minority concept.
- Group Relative Policy Optimization (GRPO): Instead of using a heavy critic model, MiPO samples a group of trajectories from the same prompt and initial noise. It normalizes the rewards within this group to compute advantages, making the reinforcement learning process highly stable and invariant to reward scaling.
- LoRA Fine-Tuning: To keep training efficient and modular, updates are applied via Low-Rank Adaptation (LoRA) adapters rather than modifying the full backbone.
- Selective Policy Optimization & KL Regularization: To prevent the model from completely forgetting how to generate majority samples (or degrading image quality), the authors apply a Kullback–Leibler (KL) penalty. Furthermore, they only update the policy during the initial 40% of the denoising timesteps, where coarse semantic and structural layouts are formed.
Datasets
- Training: MiPO uses over 100k text prompts from the HPSv2 dataset (curated from DiffusionDB and MSCOCO). No images are used during training.
- Evaluation:
- Zero-shot Classification: CIFAR10, Tiny-ImageNet, and robustness benchmarks including CIFAR10-C, Caltech, and SUN09.
- Minority Image Generation: Evaluated using 1,000 random captions from the MS-COCO validation set.
Key Insights and Ideas
- "What I cannot create, I do not understand": Borrowing from Richard Feynman, the authors demonstrate a direct, causal link between a diffusion model's generative coverage and its classification perception. Fixing generation directly fixes classification.
- Image-Free Self-Improvement: MiPO doesn't need a dataset of target images or a massive foundation model (like CLIP) to act as a reward function. The model uses its own internal denoising uncertainty as the reward signal.
- Prompt-Adaptive Generation: Unlike previous state-of-the-art methods (like MinorityPrompt) that require slow, per-prompt optimization at inference time, MiPO internalizes minority concepts into a LoRA adapter. Once trained, it generates minority-rich images instantly.
Key Results
- Classification Boosts: MiPO consistently improved zero-shot classification across multiple backbones. On Stable Diffusion 1.5, it achieved accuracy gains of up to 3.8% on standard benchmarks and 2.0% on corrupted datasets (CIFAR10-C).
- Targeted Minority Improvement: When isolating the most difficult "minority" test images in CIFAR10, MiPO boosted accuracy from 73.07% to 81.68% (an 8.61% absolute jump).
- Inference Speedup: For minority image generation, MiPO runs at the same speed as standard DDIM (1 second per image), whereas the previous best method, MinorityPrompt, takes 63 seconds per image due to test-time optimization.
- Aesthetic Preservation: Human preference evaluations (via ImageReward) showed that MiPO actually improved overall image quality scores compared to the baseline, avoiding the visual degradation typical of aggressive minority-sampling techniques.
Limitations and Future Work
While MiPO excels on clean benchmarks, it struggles on complex, cluttered, or inconsistently annotated datasets like LabelME and VOC2007, where multi-object scenes introduce too much uncertainty into zero-shot evaluation.
Additionally, there is an inherent trade-off between pushing a model to explore extreme minority concepts and maintaining the high-level aesthetic priors of the original pretrained model. Striking the perfect balance remains an open challenge for future research.
Summary updated