REST: Performance Improvement of a Black Box Model via RL-based Spatial Transformation

Jae-Myung Kim, Hyungjin Kim, Chanwoo Park, Jungwoo Lee

Abstract

Deep neural networks (DNNs) often make overconfident yet incorrect predictions on out-of-distribution samples, posing a challenge for real-world deployments. When models are deployed as black boxes, retraining is often impossible. We propose the REinforcement Spatial Transform learner (REST), a module that transforms distorted input data into samples regarded as in-distribution by black-box models. By training only the REST module using reinforcement learning and a confidence score reward, we improve robustness to geometric transformations without retraining the original model.

Introduction

Deep learning models are notoriously brittle when faced with out-of-distribution (OOD) data. For instance, a classifier trained on centered images may fail if the input is tilted or scaled. In many practical scenarios, these models are "black boxes"—proprietary or legacy systems where internal weights and gradients are inaccessible. Standard solutions like Spatial Transformer Networks (STNs) typically require end-to-end retraining, which is not feasible for black boxes. REST addresses this by acting as a pre-processing "un-distorter" that learns to map real-world distorted data back into the training distribution the black box expects.

Methodology: The REST Framework

The REST module is placed in front of a black-box image classifier and operates through an iterative reinforcement learning loop.

The REST architecture: (a) general framework and (b) detailed RL-based spatial transform learner.

Architecture

REST consists of two primary components:

  1. RL Module: An Actor-Critic architecture trained via Proximal Policy Optimization (PPO). It observes the current image state and predicts transformation parameters (rotation, scaling, shearing, and translation).
  2. Warp Module: A differentiable module that applies the predicted transformation to the image.

The Iterative Loop

Unlike one-shot transformers, REST works iteratively. It transforms the image, passes it to the black box, and receives a confidence score. This score serves as the reward signal for the RL agent. The goal is to maximize this confidence, effectively "moving" the image back into the distribution the black box was originally optimized for.

Iterative transformation process where REST increases the black-box model's confidence score over multiple steps.

Experimental Evaluation

The researchers evaluated REST across several benchmarks, including MNIST, SVHN, CIFAR10, and STL10, modified with geometric distortions of increasing complexity (Rotation, Scaling, Shearing, and Translation).

Key Results

  • Robustness to Complexity: On the most complex MNIST distortions (RSST), the baseline black-box model's accuracy was only 17.99%. Adding REST boosted this to 85.05%. Similarly, SVHN accuracy improved from 26.09% to 83.82%.
  • Generalization: REST demonstrated superior generalization compared to standard CNNs and STNs. In tests where training and test transformations were disjoint (e.g., training on "rotate right" and testing on "rotate left"), REST maintained high performance by learning a general policy of exploration rather than memorizing specific distortions.
  • Sample Efficiency: REST can learn effective strategies with as few as 1,000 distorted samples, making it significantly more efficient than standard data augmentation techniques.

Discussion and Conclusion

REST provides a "zero-touch" improvement for black-box models, requiring no access to internal parameters. Its primary strength lies in its ability to learn a transformation policy that generalizes to unseen geometric variations.

However, challenges remain. The iterative nature of REST introduces inference latency, as it may take multiple steps to "fix" an image. Additionally, if a black-box model is overconfident on incorrect OOD predictions, the RL agent may receive misleading rewards. Future work could extend this approach to non-spatial distortions like denoising or deblurring, further enhancing the robustness of black-box deployments.

Summary updated

Abstract

In recent years, deep neural networks (DNN) have become a highly active area of research, and shown remarkable achievements on a variety of computer vision tasks. DNNs, however, are known to often make overconfident yet incorrect predictions on out-of-distribution samples, which can be a major obstacle to real-world deployments because the training dataset is always limited compared to diverse real-world samples. Thus, it is fundamental to provide guarantees of robustness to the distribution shift between training and test time when we construct DNN models in practice. Moreover, in many cases, the deep learning models are deployed as black boxes and the performance has been already optimized for a training dataset, thus changing the black box itself can lead to performance degradation. We here study the robustness to the geometric transformations in a specific condition where the black-box image classifier is given. We propose an additional learner, \emph{REinforcement Spatial Transform learner (REST)}, that transforms the warped input data into samples regarded as in-distribution by the black-box models. Our work aims to improve the robustness by adding a REST module in front of any black boxes and training only the REST module without retraining the original black box model in an end-to-end manner, i.e. we try to convert the real-world data into training distribution which the performance of the black-box model is best suited for. We use a confidence score that is obtained from the black-box model to determine whether the transformed input is drawn from in-distribution. We empirically show that our method has an advantage in generalization to geometric transformations and sample efficiency.
BibTeX
@inproceedings{kim2020rest,
  author = {Jae-Myung Kim and Hyungjin Kim and Chanwoo Park and Jungwoo Lee},
  title = {{REST: Performance Improvement of a Black Box Model via RL-based Spatial Transformation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i07.6786},
}
Figure