Introduction
Convolutional Neural Networks (CNNs) have achieved remarkable success on standard image recognition benchmarks, often rivaling human performance. However, these models remain notoriously "brittle" when faced with out-of-distribution data or semantic perturbations. To bridge the gap between high benchmark accuracy and real-world reliability, we present Perturber, a web-based interactive tool designed to "stress-test" CNNs in real-time through 3D scene manipulation.
The Challenge of Model Brittleness
Traditional robustness testing is often an offline, batch-oriented process. Researchers generate perturbed datasets, run inference, and analyze aggregate metrics hours later. This latency hinders the development of an intuitive understanding of model failure modes. A model might correctly classify a dog in a standard pose but fail when the camera angle shifts or the background changes. Disentangling whether a model relies on the geometric shape of an object or its surface texture is difficult without fluid, hypothesis-driven exploration.
Methods: The Perturber Architecture
Perturber addresses these challenges by providing a visual analytics (VA) playground built on three core technical pillars:
- 3D Scene Manipulation: Utilizing Three.js, the tool renders 3D objects (e.g., dogs, cats, vehicles), allowing users to independently control geometry (rotation, zoom), scene parameters (lighting, background), and shape (morphing).
- Real-Time Inference: By leveraging TensorFlow.js, Perturber executes model inference directly in the browser. This enables a fluid feedback loop where top-5 predictions update instantly as users adjust sliders.
- Neuron Activation Views: To provide transparency, the tool visualizes internal feature maps. It combines feature visualizations (synthetic images representing what a neuron "prefers") with live activation maps to show which parts of the input are triggering specific internal representations.
Key Insights: Texture Bias and the Robustness Paradox
Interactive exploration with Perturber has revealed critical vulnerabilities in standard CNN architectures that are often obscured by aggregate metrics.
The Texture Bias
A prominent finding is the "texture bias" of standard CNNs. When the texture is removed from a 3D dog model—leaving only a smooth silhouette—standard models often misclassify the object as a "piggy bank." In contrast, robust variants (such as shape-biased or adversarially trained models) are more likely to maintain a canine-related classification, demonstrating a more human-like reliance on geometry.
The Robustness Paradox
Counter-intuitively, models trained to be robust against pixel-level adversarial noise often become more vulnerable to semantic changes. Perturber allows researchers to observe that these "robust" models can be flipped by simple yaw rotations or background swaps that do not affect standard models as severely. This suggests that current robustification techniques may trade one type of vulnerability for another.
Results and Evaluation
Perturber achieves high performance, maintaining up to 40+ FPS on modern hardware, which is essential for effective manual exploration. In case studies with machine learning experts, the tool successfully helped researchers replicate known literature findings and generate new hypotheses about model behavior.
Furthermore, quantitative verification using the Background Challenge dataset confirmed that observations made within the tool generalize: "robustified" networks achieved only 12.3% accuracy compared to 22.3% for standard models when backgrounds were varied, confirming the tool's ability to highlight real-world performance gaps.
Conclusion and Future Work
Perturber demonstrates the power of interactive visual analytics in uncovering the hidden vulnerabilities of CNNs. By moving from static evaluation to interactive manipulation, researchers can better understand and eventually mitigate the brittleness of modern computer vision models. Future work will focus on automated neuron discovery to highlight significant activation changes and extending support to encoder-decoder architectures used in tasks like semantic segmentation.
Summary updated