Towards an Understanding of Neural Networks in Natural-Image Spaces

Yifei Fan, Anthony J. Yezzi

TL;DR

This paper argues that dataset bias and adversarial examples are not just bugs, but intrinsic properties of how neural networks learn in "natural-image spaces." By modeling these spaces as discrete "quotient spaces" and viewing training through variational calculus, the authors demonstrate that not all data is beneficial and that adding a "random noise" category can actually improve model robustness and accuracy.

What Problem Does This Paper Address?

Despite their success, deep neural networks (DNNs) suffer from two major "uncertainties": dataset bias (failing when moved from one dataset to another) and adversarial examples (tiny, invisible changes to an image that cause misclassification).

The authors suggest that the common "more data is always better" mantra is flawed. They address the lack of a theoretical framework that explains why these issues occur based on the geometric and topological properties of the space of natural images. Specifically, they tackle the "zero-constrained" nature of training—where a network is only told what a function should be at specific training points, leaving it to make arbitrary (and often fragile) "guesses" everywhere else.

Methods

The authors propose a multi-pronged approach to understanding and improving DNN performance:

  1. Variational Perspective: They treat the target function as an extremum point in an infinite-dimensional space. They argue that because training data is finite, the resulting learned function is "zero-constrained," meaning its behavior between data points is essentially undefined and prone to warping.
  2. Point-function Duality: This concept explains that while we want to learn a continuous function, we are actually just fitting points. This leads to "illusive" training samples that force the network to deform its decision boundaries in unnatural ways to minimize loss.
  3. Quotient Space Model: Instead of assuming all images of a "cat" are connected in a smooth manifold, they model the natural-image space as a collection of separate "equivalence classes" (e.g., an image and its filtered versions).
  4. Random-Noise Category: To break the "winner-takes-all" logic where a network must pick one of the provided labels, they introduce an additional "random-noise" class during training. This teaches the network what a natural image is not.
  5. Illusive Sample Detection: They use higher-order statistics from the training process to identify "illusive samples"—data points that are consistently misclassified—and experiment with excluding them to prevent overfitting.

Datasets

  • CIFAR-10: The primary dataset used for retraining experiments and evaluating the impact of noise and illusive samples.
  • AlexNet (Pre-trained): Used to test "hybrid images" (combining high-frequency components of one image with low-frequency components of another).
  • CleverHans: Used as a benchmark to evaluate robustness against adversarial attacks.

Key Insights and Ideas

  • The "Sherlock Holmes" Logic: Networks often misclassify because they are forced to choose the "least impossible" label from a limited set. If a network sees something it doesn't recognize, it has no "none of the above" option, leading to high-confidence errors.
  • Data is Not Created Equal: The authors show that some individual data points have a "disproportionate and counterproductive influence." Removing these "illusive" samples can actually help the network generalize better.
  • Connectivity vs. Slicing: Standard networks try to draw smooth, connected decision boundaries. However, natural images of the same category might exist in separate, disconnected "islands" in high-dimensional space. Forcing these islands to connect creates "bridges" through regions of non-natural images, which are exactly where adversarial examples tend to hide.
Figure 3: Intuitive illustration on the connectivity of natural image space. Points represent natural images.
  • Intrinsic Bias: Dataset bias isn't just about poor data collection; it's an intrinsic property of point-function duality. As seen in Figure 1, fitting the same underlying function with different sample distributions leads to wildly different boundary geometries.
Figure 1: Training samples affect the geometry of the approximated space.

Key Results

  • Noise as Information: Adding a "random-noise" category to CIFAR-10 training provided an incremental boost to test accuracy (e.g., improving from ~70.8% to ~71.4% in certain subsets) and made the classifier more "determinant" (higher confidence in correct predictions).
  • Alleviating Overfitting: By detecting and excluding "illusive" training samples, the authors successfully reduced overfitting. Figure 5 shows that training without these samples leads to a more stable accuracy curve.
Figure 5: Training without consistently misclassified samples help prevent over-fitting.
  • Adversarial Robustness: Retraining without illusive samples improved the average accuracy on adversarial examples (using the CleverHans library) by several percentage points across various perturbation levels ($\epsilon$).
  • Quotient Space Potential: The authors demonstrate that "slicing" the equivalence classes—treating illusive samples as a new sub-label and then "gluing" them back together with a look-up table—can potentially increase test accuracy by up to 3% without additional hardware.

Limitations and Future Work

The authors acknowledge that their "quotient space" model is currently more of a conceptual framework than a fully automated system. Identifying "illusive samples" requires an initial training phase to gather statistics, which adds computational overhead.

Future work includes developing better ways to measure the "value" or "impact" of individual training samples and creating more rigorous mathematical models for the topological properties of natural-image spaces. They also suggest that adding more "negative" training categories (beyond just random noise) could further help networks handle exceptions and outliers.

Summary updated

Abstract

Two major uncertainties, dataset bias and adversarial examples, prevail in state-of-the-art AI algorithms with deep neural networks. In this paper, we present an intuitive explanation of these issues as well as an interpretation of the performance of deep networks in a natural- image space. The explanation consists of two parts: the variational-calculus view of machine learning and a hypothetical model of natural-image spaces. Following the explanation, we (1) demonstrate that the values of training samples differ, (2) provide incremental boosts to the accuracy of a CIFAR-10 classifier by introducing an additional "random-noise" category during training, and (3) alleviate over-fitting thereby enhancing the robustness of a classifier against adversarial examples by detecting and excluding illusive training samples that are consistently misclassified. Our overall contribution is therefore twofold. First, while most existing algorithms treat data equally and have a strong appetite for more data, we demonstrate in contrast that an individual datum can sometimes have disproportionate and counterproductive influence, and that it is not always better to train neural networks with more data. Next, we consider more thoughtful strategies by taking into account the geometric and topological properties of natural-image spaces to which deep networks are applied.
BibTeX
@inproceedings{fan2019towards,
  author = {Yifei Fan and Anthony J. Yezzi},
  title = {{Towards an Understanding of Neural Networks in Natural-Image Spaces}},
  booktitle = {Workshops @ CVPR},
  year = {2019},
  url = {https://openreview.net/forum?id=if-d63033XI},
}
Figure