TL;DR
OmniInput is a model-centric evaluation framework that assesses neural classifiers across their entire input space rather than relying on curated, fixed test sets. By coupling Wang–Landau physical sampling algorithms with selective human annotation, it constructs comprehensive precision-recall curves that expose severe overconfidence and hidden failure modes missed by standard benchmarks.
What Problem Does This Paper Address?
Traditional machine learning evaluations are data-centric: models are graded against static, human-annotated test sets. However, strong in-distribution (and even curated out-of-distribution) test accuracy often hides brittle failure modes. Deep networks are notorious for overconfident predictions, frequently assigning near-100% confidence to uninformative noise, adversarial perturbations, or nonsensical inputs.
Furthermore, out-of-distribution (OOD) benchmarks are inconsistent: a model's ranking can fluctuate wildly depending on which arbitrary OOD dataset is selected. The authors tackle this blind spot by proposing a way to evaluate models over their entire input space ($\Omega = \{0, \dots, N\}^D$, which for even a simple $28 \times 28$ 8-bit image contains $256^{784}$ possible inputs—far exceeding the number of atoms in the universe).
Methods
OmniInput shifts evaluation from data-centric to model-centric using four primary steps:
- Output Distribution Sampling: The framework treats the model's output distribution $\rho(z)$—the frequency of all possible inputs mapping to logit $z$—similarly to the physical "density of states." Using the Gradient Wang–Landau (GWL) sampler, it uniformly navigates the model’s logit space, capturing representative input samples across discrete output bins.
- Selective Annotation: Evaluators inspect a small set of representative inputs sampled per bin (e.g., 40–50 samples) and assign a score $r(z) \in [0, 1]$ indicating whether the input matches human judgment for the target class.
- Distribution-Weighted Metrics: Precision and unnormalized recall are estimated for any logit threshold $\lambda$ by weighting bin scores against the global output distribution: $$\text{precision}_\lambda = \frac{\sum_{z \ge \lambda} r(z)\rho(z)}{\sum_{z \ge \lambda} \rho(z)}, \quad \text{recall}_\lambda \propto \sum_{z \ge \lambda} r(z)\rho(z)$$
- Full-Space Precision-Recall Curves: Sweeping $\lambda$ generates a precision-recall curve representing performance over the complete input space.
[GWL Sampler] ---> [Output Distribution ρ(z) & Representative Inputs]
---> [Human Annotation r(z)]
---> [Weighted Full-Space PR Curve]
Datasets
The authors evaluated models across both synthetic full spaces and traditional datasets:
- MNIST (0 vs. 1): Classifiers and energy-based generative models evaluated over the full $256^{28 \times 28}$ image space.
- OOD Benchmarks (for comparison): Fashion-MNIST, Kuzushiji-MNIST, EMNIST (byclass split), and Q-MNIST.
- CIFAR-10 (Airplane vs. Automobile): ResNet binary classifier evaluated over the discrete $32 \times 32 \times 3$ image space.
- SST2: DistilBERT sentiment classifier evaluated over fixed-length token sequences (lengths 10 and 66).
Key Insights and Ideas
- Standard Benchmarks Can Hide Catastrophic Flaws: Models achieving >97% or even >99% test accuracy on MNIST can have near-zero precision over the full input space. For example, a vanilla CNN learned to classify almost any dark-background noise pattern as the digit "1."
- Data Augmentation Beats Pure Generative Priors in Recall: While energy-based generative models produced realistic digits at high logits, they exhibited low recall. Conversely, training with uniform noise data augmentation (
RES-AUG-MNIST-0/1) allowed the model to maintain high precision and substantially broader recall. - Generative Evaluation Metrics (like FID) Fail in OOD Spaces: Automated metrics like Fréchet Inception Distance (FID) gave misleadingly favorable scores to random noise outputs because feature extractors (e.g., Inception) are themselves uncertified outside their training distribution.
- Convergence of Optimal Classifiers and Generators: The authors speculate that a theoretically perfect open-world classifier and a perfect generative model converge to the same functional mapping.
Key Results
- Inconsistent Traditional Rankings: On pre-defined test sets, model rankings reversed depending on the dataset. A generative model (
RES-GEN-MNIST-1) ranked 3rd on in-distribution test data (99.99% AUPR) but 1st across all OOD sets (e.g., 100.00% on Fashion-MNIST, 99.87% on EMNIST). - OmniInput PR Performance: OmniInput revealed that
RES-AUG-MNIST-0/1was the superior model overall. In contrast, standard CNNs (CNN-MNIST-0/1andCNN-AUG-MNIST-0/1) exhibited precision near 0.0 across high logits due to massive overconfidence on non-digit noise. - CIFAR-10 & DistilBERT Failures:
- A ResNet on CIFAR-10 with 93.34% standard accuracy yielded virtually 0 precision across its 299 sampled bins.
- DistilBERT on SST2 (91% accuracy) scored near-zero precision on full sequence spaces, picking up isolated positive keywords while remaining oblivious to syntax or coherence.
- Low Human Annotation Burden: Annotation stabilized rapidly; only 40 to 50 samples per bin were necessary for the precision-recall curve to converge.
Limitations and Future Work
- Sampler Scalability: The framework relies heavily on advanced MCMC algorithms (like GWL). Sampling the output distribution of massive vision models or modern Large Language Models (LLMs) remains computationally demanding.
- Task Scope: Experiments are currently limited to binary classification and short text sequence lengths.
- Future Directions: Developing faster, massively parallel samplers and extending OmniInput to multi-class and multi-dimensional output spaces.
Summary updated