Is Ordered Weighted ℓ1 Regularized Regression Robust to Adversarial Perturbation? A Case Study on OSCAR

Pin-Yu Chen, Bhanukiran Vinzamuri, Sijia Liu

Abstract

While adversarial attacks are well-documented in image classification, their impact on regression models remains under-explored. This study investigates the robustness of Ordered Weighted ℓ1 (OWL) regularized regression, specifically the OSCAR model, against targeted adversarial perturbations. By formulating the attack as an optimization problem, we demonstrate that small, norm-bounded noise can significantly disrupt the feature-grouping capabilities of OSCAR. Our findings reveal a critical robustness gap between average-case random noise and worst-case adversarial noise in sparse regression.

Introduction

Regression is a cornerstone of data science, yet its vulnerability to adversarial manipulation is less understood than that of classification. This research focuses on Ordered Weighted ℓ1 (OWL) regularized regression, specifically the Octagonal Shrinkage and Clustering Algorithm for Regression (OSCAR). OSCAR is designed to handle highly correlated covariates by grouping them with identical coefficients. While previous theoretical work established error bounds for these models under random noise, this paper explores whether an adversary can intentionally craft a small amount of noise to break the model’s ability to identify correct feature groups.

Methodology

The authors treat the search for adversarial noise as a constrained optimization problem. The threat model assumes an adversary adds a noise vector $\nu$ to measurements ($y = Ax^* + \nu$), where the noise is budget-constrained ($||\nu||_1 \leq \epsilon$). The objective of the attacker is to find a $\nu$ that maximizes the distance between the model's predicted coefficients $\hat{x}$ and the true coefficients $x^*$.

To solve the regression, the authors utilize the Approximate Proximity Operator (APO) for OSCAR. The adversarial noise is identified by reformulating the attack as a LASSO-style problem and solving it using the Iterative Shrinkage-Thresholding Algorithm (ISTA). A key technical contribution is the explicit derivation of the gradient of the regression loss with respect to the noise, allowing the error to be "backpropagated" to find the most damaging perturbations.

Experimental Setup

The study utilizes a synthetic dataset with $p = 100$ features and $n = 50$ instances. The features are generated with a pre-defined grouping structure where certain features are highly correlated and should be treated as a block. Synthetic data is preferred in this context as it provides a clear "ground truth," making it easy to quantify how much an attack misaligns the model's coefficient estimates.

Results and Analysis

Experimental results demonstrate that OSCAR is highly sensitive to targeted perturbations. As the attack budget $\epsilon$ increases from 0.05 to 0.3, the model's ability to identify correct feature groups degrades significantly.

Comparison of regression coefficients for Ground Truth, standard OSCAR, and OSCAR under adversarial attack across varying noise levels.

As shown in the figure above, even at low noise levels ($\epsilon = 0.05$), feature groups that should have identical coefficients (forming a flat line) begin to "splinter." At higher budgets, the regression results deviate drastically from the ground truth. This "robustness gap" suggests that models performing well under random Gaussian noise may still fail catastrophically under strategic attacks. The very clustering mechanism that makes OSCAR valuable becomes a specific target for the adversary, leading to incorrect feature identification.

Conclusion

This case study on OSCAR highlights the vulnerability of OWL-regularized regression to adversarial attacks. The results suggest that theoretical stability under random noise does not guarantee robustness against worst-case perturbations. Future work includes extending this framework to the broader OWL family, exploring black-box attack scenarios, and developing defense mechanisms such as adversarial training to enhance the resilience of these models in critical scientific and financial applications.

Summary updated

BibTeX
@inproceedings{chen2018is,
  author = {Pin-Yu Chen and Bhanukiran Vinzamuri and Sijia Liu},
  title = {{Is Ordered Weighted ℓ1 Regularized Regression Robust to Adversarial Perturbation? A Case Study on OSCAR}},
  booktitle = {GlobalSIP},
  year = {2018},
  doi = {10.1109/globalsip.2018.8646623},
}
Figure