Towards an Understanding of Neural Networks in Natural-Image Spaces

Yifei Fan, Anthony J. Yezzi

Abstract

Despite the remarkable success of deep neural networks (DNNs), they remain susceptible to dataset bias and adversarial examples. This paper proposes that these issues are not mere bugs but intrinsic properties of how networks learn in "natural-image spaces." By framing training through the lens of variational calculus and modeling image spaces as discrete quotient spaces, the authors demonstrate that the "zero-constrained" nature of finite data leads to warped decision boundaries. To mitigate this, they introduce a "random-noise" category and a method for filtering "illusive samples," resulting in improved model robustness and generalization.

Introduction

The prevailing paradigm in deep learning suggests that "more data is always better." However, DNNs frequently fail when transitioning between datasets or when faced with imperceptible adversarial perturbations. This research addresses the lack of a theoretical framework explaining these failures by examining the geometric and topological properties of the space of natural images. The authors argue that because networks are only constrained at specific training points, their behavior in the vast regions between these points is essentially undefined, leading to fragile and unnatural decision boundaries.

Theoretical Framework: Point-Function Duality

A core contribution of this work is the concept of point-function duality. In a variational perspective, the target function learned by a network is an extremum point in an infinite-dimensional space. However, because training data is finite, the resulting function is "zero-constrained."

Figure 1: Training samples affect the geometry of the approximated space.

As illustrated in Figure 1, fitting a function to discrete points allows for significant "warping" between those points. This warping is the root of dataset bias; different distributions of training samples for the same underlying function will result in wildly different boundary geometries, even if they achieve similar training loss.

The Quotient Space Model and "Illusive Samples"

Rather than assuming all images of a category (e.g., "cat") exist on a single smooth manifold, the authors model the natural-image space as a collection of separate "equivalence classes"—a quotient space.

Standard training forces networks to draw connected decision boundaries between these potentially disconnected "islands" of data. This creates "bridges" through regions of non-natural images, which are precisely where adversarial examples reside. Within this framework, the authors identify illusive samples: data points that force the network to deform its boundaries in unnatural ways to minimize loss. These samples often have a disproportionate and counterproductive influence on generalization.

Addressing the "Sherlock Holmes" Logic

Neural networks often operate under a "winner-takes-all" logic, where they must assign one of the provided labels to any input. The authors describe this as "Sherlock Holmes" logic: the network chooses the "least impossible" label from its limited set, even if the input is clearly not a natural image of that class.

Figure 6: A submarine misclassified as a great white shark, illustrating the 'least impossible' label logic.

As seen in Figure 6, a network may misclassify a submarine as a "great white shark" simply because it has no "none of the above" option. To combat this, the authors introduced a random-noise category during training. This teaches the network what a natural image is not, providing a rejection class that improves both test accuracy and prediction confidence.

Experimental Results and Robustness

Experiments conducted on the CIFAR-10 dataset and with pre-trained AlexNet models yielded several key insights:

  1. Noise as Information: Adding the random-noise class provided an incremental boost to test accuracy (e.g., from ~70.8% to ~71.4%) and increased the determinancy of the classifier.
  2. Alleviating Overfitting: By using higher-order statistics to detect and exclude "illusive samples," the authors reduced overfitting and created more stable accuracy curves during training.
  3. Adversarial Robustness: Retraining without illusive samples improved accuracy on adversarial benchmarks (via the CleverHans library) by several percentage points across various perturbation levels.

Conclusion

This research suggests that the path to more robust AI lies in understanding the topological constraints of natural-image spaces. By moving away from the "zero-constrained" point-fitting model and incorporating negative information (like noise) and sample filtering, we can develop networks that generalize more naturally and resist adversarial manipulation. Future work will focus on automating the detection of illusive samples and refining the mathematical models for quotient spaces.

Summary updated

BibTeX
@inproceedings{fan2019towards,
  author = {Yifei Fan and Anthony J. Yezzi},
  title = {{Towards an Understanding of Neural Networks in Natural-Image Spaces}},
  booktitle = {Workshops @ CVPR},
  year = {2019},
  url = {https://openreview.net/forum?id=if-d63033XI},
}
Figure