TL;DR
This paper introduces SPARTA, a novel black-box adversarial paraphrasing method that exposes the fragility of reasoning segmentation models. By optimizing prompts within the continuous latent space of a text autoencoder using reinforcement learning, SPARTA generates grammatically correct, semantically equivalent queries that severely degrade model performance, outperforming previous text attacks by up to 2×.
What Problem Does This Paper Address?
Multimodal Large Language Models (MLLMs) have made massive strides in vision-language tasks like reasoning segmentation—where a model must output a precise binary segmentation mask based on an implicit, complex text query (e.g., "What might the young girl be using to eat her dessert?").
While previous robustness research heavily focused on perturbing the visual input (like adding pixel noise), the textual input has remained largely underexplored. In real-world applications, users express the same intent in countless different ways. A robust model should produce the same segmentation mask regardless of how a prompt is paraphrased. This paper tackles the challenge of adversarial paraphrasing: generating fluent, grammatically correct text variations that preserve the original meaning but completely break the segmentation model's performance.
Methods
To circumvent the discrete optimization problem of text tokens—which often causes white-box gradient attacks to output ungrammatical gibberish—the authors developed SPARTA (Sentence-level Perturbation Adversarial Text Attack).
Key components of the method include:
- Latent Sentence Space Exploration: SPARTA maps the input query into a continuous, low-dimensional semantic latent space using a pretrained text autoencoder called SONAR (a 1B-parameter multilingual model).
- Reinforcement Learning Optimization: Using Proximal Policy Optimization (PPO), SPARTA learns a stochastic policy to perturb the latent vector.
- Targeted Reward Function: The policy is rewarded for minimizing the Intersection over Union (IoU) between the predicted mask and the ground truth (maximizing degradation) while maintaining a regularization penalty to keep the perturbed vector close to the original embedding space.
- Rigorous Post-Hoc Filtering: Because automated evaluation of paraphrases is notoriously difficult, the authors designed a hybrid validation protocol. It combines an LLM grader (Qwen3-32B) with regular expressions (to enforce capitalization and punctuation consistency) and cosine similarity filtering (via Qwen3-Embedding-8B) to ensure the generated text is a valid paraphrase.
Datasets
The authors evaluated their approach on two primary reasoning segmentation benchmarks:
- ReasonSeg: The standard benchmark dataset for evaluating reasoning-based segmentation tasks.
- LLMSeg-40k: A newer, larger-scale dataset collected via ChatGPT-4, featuring longer queries (averaging 15.2 words) and significantly higher linguistic complexity.
Due to computational constraints, a subset of 300 samples from each dataset was utilized for the evaluation.
Key Insights and Ideas
- Latent Space vs. Token Substitution: Operating in a continuous latent space allows the attack to modify sentence structure globally rather than relying on simple word-swapping or character edits, leading to much more natural-sounding paraphrases.
- The Length Confound in Embeddings: During their analysis of the SONAR autoencoder, the authors discovered that a single dimension (dim 654) strongly correlates with sentence length ($r = -0.956$). This means text autoencoders explicitly encode sequence length in a disentangled coordinate, which can inadvertently confound semantic similarity metrics if sentences vary in length.
- LLM Evaluation Blindspots: The study revealed that modern LLMs are overly permissive when judging paraphrases, often rating overly verbose, riddle-like sentences as valid. This highlighted the necessity of their hybrid evaluation protocol (LLM + RegExp + Cosine Similarity).
Key Results
SPARTA demonstrated a remarkable ability to expose vulnerabilities across three state-of-the-art reasoning segmentation models (LISA, LISA++, and GSVA).
- Performance Boost: SPARTA consistently outperformed competitive baselines (like GBDA and PAIR), achieving up to a 2× improvement in attack success rate metrics on the LLMSeg-40k dataset.
- Model Vulnerability: When combining all baselines into a "unified attack," the success rate reached up to 68% at a 10% relative IoU drop threshold. This proves that current reasoning segmentation models remain highly vulnerable to well-crafted, semantic-preserving text adjustments.
- Capacity and Robustness: Larger models (e.g., LISA 13B vs. 7B) consistently demonstrated higher resistance to adversarial paraphrasing, suggesting that scaling model capacity inherently aids textual robustness.
Limitations and Future Work
Despite its success, the authors note a few limitations:
- Post-Hoc Validity: SPARTA does not guarantee that a valid paraphrase is generated during the optimization loop; invalid or unnatural prompts must be filtered out after generation. Future work could integrate validity constraints directly into the RL training process.
- Human Naturalness: Some generated text variations can still sound slightly unnatural to human users due to inherent limitations in current text autoencoders.
- Lack of Defenses: The paper focuses exclusively on the attack vector. Developing robust defense strategies and training techniques to mitigate these paraphrasing vulnerabilities remains an open and critical next step.
Summary updated