Towards Automated Testing and Robustification by Semantic Adversarial Data Generation

Rakshith Shetty, Mario Fritz, Bernt SchieleECCV 2020 · OpenReview

Abstract

Widespread application of computer vision systems in real world tasks is currently hindered by their unexpected behavior on unseen examples. This occurs due to limitations of empirical testing on finite test sets and lack of systematic methods to identify the breaking points of a trained model. In this work we propose semantic adversarial editing, a method to synthesize plausible but difficult data points on which our target model breaks down. We achieve this with a differentiable object synthesizer which can change an object’s appearance while retaining its pose. Constrained adversarial optimization of object appearance through this synthesizer produces rare/difficult versions of an object which fool the target object detector. Experiments show that our approach effectively synthesizes difficult test data, dropping the performance of YoloV3 detector by more than 20 mAP points by changing the appearance of a single object and discovering failure modes of the model. The generated semantic adversarial data can also be used to robustify the detector through data augmentation, consistently improving its performance in both standard and out-of-dataset-distribution test sets, across three different datasets.

Introduction

Deep learning models for computer vision are notoriously brittle. While they perform well on standard test sets, they often fail in the real world when encountering "out-of-distribution" (OOD) scenarios—such as a car in an unusual color or a person in a strange pose. Current testing methodologies rely on finite, static datasets that cannot capture the "long tail" of possible real-world variations. Furthermore, traditional adversarial attacks usually involve adding imperceptible pixel-level noise, which fails to represent meaningful semantic changes like lighting, color, or texture variations that occur in reality.

Conceptual overview of semantic adversarial manipulation: moving an object's appearance along the manifold until the model's decision boundary is crossed.

To address this, we introduce semantic adversarial editing, a method to automatically identify and rectify model "breaking points." By creating label-consistent but difficult semantic variations, we can stress-test and improve detectors in a systematic way.

Methodology

The proposed pipeline consists of two primary components: a differentiable object synthesizer and a semantic adversarial optimization framework.

The Object Synthesizer

The synthesizer is designed to disentangle an object's shape from its appearance. It utilizes a Shape Encoder with a "binary part segmentation bottleneck" to capture complex geometries (e.g., the frame of a bicycle). Simultaneously, an Appearance Encoder extracts latent codes representing the visual characteristics of different object parts. A Decoder then combines these features to reconstruct the image. This disentanglement allows the model to generate variations that maintain the original scene's context and pose while altering visual attributes.

Semantic Adversarial Optimization

To identify failure modes, the system takes a real image, removes a target object via in-painting, and uses the synthesizer to re-insert a modified version. We optimize the appearance latent codes to maximize the detector's loss. To ensure the resulting images remain realistic and label-consistent, we constrain the optimized appearance to stay within the convex hull of real-world "guiding templates"—existing examples of that object class.

Results and Discussion

We evaluated our approach across major benchmarks including MS COCO, Pascal VOC, and BDD100k, as well as OOD datasets like UnRel and D2-City.

Effectiveness as an Attacker

By altering the appearance of just a single object in an image, our method dropped the performance of a YOLOv3 detector by over 20 mAP points. A human validation study confirmed that in 93% of these "failure" cases, humans could still correctly identify the object, proving that the changes are semantic and preserve the ground truth label.

Iterative semantic adversarial optimization showing how objects like birds, balls, and dogs are modified over iterations to fool the detector.

Failure Mechanisms

Our analysis identified four primary reasons for detector failure:

  1. Camouflaging: Objects are modified to blend into the background.
  2. Occlusion: Slight shifts in object placement hide critical features.
  3. Texture Over-reliance: Detectors often prioritize specific colors/textures over shape (e.g., a white cow being misidentified as a sheep).
  4. Contextual Overriding: Strong surrounding context (e.g., a field of sheep) causes the model to ignore object-specific features if they are slightly atypical.

Robustification

Using these generated "hard positives" for data augmentation consistently improved model performance. For instance, on the OOD UnRel dataset, the robustified COCO model achieved a 1.8 mAP improvement, demonstrating enhanced generalization to the "long tail" of real-world data.

Conclusion

Semantic adversarial editing provides a powerful tool for both testing and improving the robustness of computer vision models. While current limitations include a fixed object pose and constraints on resolution (128x128), the method successfully bridges the gap between pixel-level noise and meaningful semantic variations. Future work will explore more complex 3D reasoning to allow for pose variations and higher-resolution synthesis.

Summary updated

Abstract

Widespread application of computer vision systems in real world tasks is currently hindered by their unexpected behavior on unseen examples. This occurs due to limitations of empirical testing on finite test sets and lack of systematic methods to identify the breaking points of a trained model. In this work we propose semantic adversarial editing, a method to synthesize plausible but difficult data points on which our target model breaks down. We achieve this with a differentiable object synthesizer which can change an object’s appearance while retaining its pose. Constrained adversarial optimization of object appearance through this synthesizer produces rare/difficult versions of an object which fool the target object detector. Experiments show that our approach effectively synthesizes difficult test data, dropping the performance of YoloV3 detector by more than 20 mAP points by changing the appearance of a single object and discovering failure modes of the model. The generated semantic adversarial data can also be used to robustify the detector through data augmentation, consistently improving its performance in both standard and out-of-dataset-distribution test sets, across three different datasets.
BibTeX
@inproceedings{shetty2020towards,
  author = {Rakshith Shetty and Mario Fritz and Bernt Schiele},
  title = {{Towards Automated Testing and Robustification by Semantic Adversarial Data Generation}},
  booktitle = {European Conference on Computer Vision},
  year = {2020},
  url = {https://openreview.net/forum?id=4RobYOEwM4},
}
Figure