DeepView: Visualizing Classification Boundaries of Deep Neural Networks as Scatter Plots Using Discriminative Dimensionality Reduction

Alexander Schulz, Fabian Hinder, Barbara Hammer

Abstract

Machine learning algorithms using deep architectures have become increasingly powerful but also more complex and difficult to comprehend. Most existing interpretation methods focus on single-input decisions. We propose DeepView, a tool that visualizes the decision function of a deep neural network (DNN) alongside the dataset in two dimensions using discriminative dimensionality reduction. This allows for the inspection of model behavior regarding outliers, adversaries, and poisoned data. Code is available at https://github.com/LucaHermes/DeepView.

Introduction

Deep neural networks (DNNs) are often criticized as "black boxes." While local explanation methods like saliency maps provide insights into individual predictions, they fail to offer a global perspective on the model's decision landscape. Understanding where class boundaries lie and how a model generalizes in regions devoid of training data is crucial for identifying vulnerabilities. The primary obstacle is the high dimensionality of the input space, which standard dimensionality reduction (DR) techniques like t-SNE or UMAP struggle to project without losing the classifier's underlying logic.

Methodology

DeepView addresses the dimensionality challenge through two primary innovations: Fisher UMAP and a learned inverse mapping.

Fisher UMAP (Discriminative DR)

Standard DR techniques typically preserve Euclidean distances, which may not reflect the model's internal logic. DeepView utilizes Fisher UMAP, which employs a metric based on the Fisher Information Matrix. Specifically, it calculates the Jensen-Shannon divergence between the probability distributions produced by the model for different inputs. This ensures that points the model perceives as similar are clustered together in the 2D projection, regardless of their raw pixel-level similarity.

Inverse Mapping and Boundary Visualization

To visualize decision boundaries, the tool must predict class labels for every point in the 2D space. DeepView implements a mathematically grounded inverse mapping ($\pi^{-1}$) that projects 2D coordinates back into the high-dimensional input space. This allows the system to "probe" the model at any pixel in the visualization, coloring the background to represent predicted classes and confidence levels (entropy).

Overview of the DeepView pipeline, showing the transition from high-dimensional data to a 2D decision landscape with clear boundaries.

Key Insights and Applications

DeepView provides a model-centric view of the data, revealing phenomena that traditional visualizations obscure.

  • Decision "Pockets" and Overfitting: The tool can identify small, isolated regions where the model's classification flips unexpectedly. These "pockets" often indicate overfitting or adversarial vulnerabilities.
  • Adversarial and Poisoning Detection: By visualizing the decision landscape, researchers can spot adversarial examples hidden within class territories. Furthermore, DeepView can separate "poisoned" samples (e.g., images with a hidden trigger) into distinct clusters, even if they appear normal to human observers.
  • Interactive Exploration: The inverse mapping enables users to click on any point in the 2D plot to see a reconstructed image representing the model's "perception" of that coordinate.
A DeepView scatter plot showing a binary classification task. The background color indicates class probability and decision boundaries.

Results

The effectiveness of DeepView was evaluated on benchmark datasets including CIFAR-10 and Fashion-MNIST. A key metric, $Q_{kNN}$, was used to measure how well the 2D projection preserves the model's labels. On CIFAR-10, Fisher UMAP achieved a score of 96.3%, significantly outperforming standard Euclidean UMAP (18.3%). This demonstrates that Fisher UMAP successfully captures the discriminative logic of the DNN. In practical tests, DeepView successfully visualized adversarial "ships" hidden within "bird" class territory, showing exactly how the decision boundary had been warped.

Conclusion

DeepView offers a powerful, global perspective on DNN behavior, complementing existing local explanation methods. While challenges remain regarding the scalability of the $O(N^2)$ distance calculations and the fidelity of inverse mapping reconstructions, it represents a significant step toward making deep learning models more transparent and robust.

Summary updated

Abstract

Machine learning algorithms using deep architectures have been able to implement increasingly powerful and successful models. However, they also become increasingly more complex, more difficult to comprehend and easier to fool. So far, most methods in the literature investigate the decision of the model for a single given input datum. In this paper, we propose to visualize a part of the decision function of a deep neural network together with a part of the data set in two dimensions with discriminative dimensionality reduction. This enables us to inspect how different properties of the data are treated by the model, such as outliers, adversaries or poisoned data. Further, the presented approach is complementary to the mentioned interpretation methods from the literature and hence might be even more useful in combination with those. Code is available at https://github.com/LucaHermes/DeepView
BibTeX
@inproceedings{schulz2020deepview,
  author = {Alexander Schulz and Fabian Hinder and Barbara Hammer},
  title = {{DeepView: Visualizing Classification Boundaries of Deep Neural Networks as Scatter Plots Using Discriminative Dimensionality Reduction}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  url = {https://openreview.net/forum?id=QHkeVnaaA3F},
}
Figure