Camouflaged animals are hard to segment because their color and coarse texture match the background, so ordinary image features see almost no edge. The physical boundary still leaves a trace in the fine-grained, high-frequency part of the image, meaning the rapid pixel-to-pixel variations a Fourier or wavelet transform separates from smooth color regions. The trouble is that large pretrained vision models tend to ignore exactly those signals, and small add-on modules trained on top of a frozen model cannot easily restore what the model never encoded. We propose to put the missing signal back into the model's input sequence as a handful of explicit "frequency tokens", short vectors that summarize how much energy each frequency band carries in each image region. These sit alongside image patches and the words of a request such as "the spotted gecko on the bark", inside one frozen transformer. Because every token attends to every other from the first layer, the words can decide which frequency bands matter for this particular target. A spotted pattern and a smooth-skinned frog leave their traces in different bands.
The main claim is that the frequency cue a camouflage model needs depends on what is being looked for. A referring model, one that segments the object named in a text query, should therefore select spectral bands per query rather than apply one fixed high-pass filter. If this holds, it gives a cheap, testable way around "representation rigidity" (the ceiling a frozen backbone imposes when its features omit what the task needs). It also yields a diagnostic: you can watch the selection change when only the words change.
Core Claim
Prior work establishes two facts and leaves a third open. Explicit Visual Prompting shows that dataset-level prompts on a frozen ViT fail on low-level tasks such as camouflage, and that injecting image-specific high-frequency components extracted by FFT fixes much of this, reaching S-measure 0.843 on COD10K with 3.7M trainable parameters. This is the motivating evidence for rigidity. However, EVP's frequency split is hand-fixed and the model is unimodal. MLKG shows that rich language about morphology and texture, produced by LLaVA and injected into a LoRA-tuned SAM, helps referring camouflage segmentation. Its pipeline is decoupled and works only on RGB pixels, so the language can describe "fine stripes" but has no direct access to a representation of fine stripes. What neither tests is whether language and explicit spectral evidence interact usefully: whether a texture word can pick out the band that carries the matching boundary. We claim it can, and that this interaction requires the spectral evidence to be present early in the shared sequence rather than fused at the decoder.
Research Program
The work has three parts, and the first can change how the rest is read.
The first part is a rigidity probe. Before building anything, we test whether high-frequency information is actually absent from frozen features or merely unused. For a frozen ViT and for SAM's image encoder, we fit linear probes at each layer to predict the local band energy of a stationary wavelet decomposition at object boundaries on COD10K. If the probes decode band energy well, the bottleneck is how the features are used rather than what they contain, and a stronger spatial adapter may suffice. We would report that result as a finding. If decodability collapses with depth, as EVP's invariance argument predicts, explicit injection is justified. Prior rigidity studies such as Lai et al. on pathology and DCPT on nighttime tracking infer rigidity from adapter failure. Neither probes the frozen features for the specific missing signal, and we do.
The second part is the model. We decompose each image with a stationary wavelet transform into $B$ bands (we expect $B=4$ to 6). We average-pool each band's magnitude over a coarse $G \times G$ grid and linearly project it to a token. The frequency token for band $b$ at region $g$ is
$$f_{b,g} = W_b \, \mathrm{pool}_g\big(|\mathcal{S}_b(x)|\big) + e_b + p_g ,$$
where $x$ is the image, $\mathcal{S}_b$ the wavelet band, $W_b$ a learned projection, and $e_b, p_g$ learned band and position embeddings. Pooling to $B G^2$ tokens (256 at $G=8$, $B=4$) keeps the sequence growth modest and avoids the quadratic blow-up that full-resolution spectral tokens would cause, a cost the file flags for early fusion. Text tokens from a frozen text encoder, projected into the visual width, are appended as well. All three token types pass through the frozen ViT, and the only trainable parts are the projections, low-rank updates to attention, and a mask decoder, kept within a 5–7% parameter budget to match EVP and FGSA-Net.
S³AM warns that uncalibrated early fusion lets noisy auxiliary high-frequency signals pollute every layer. We therefore gate each frequency token by a scalar computed from the pooled text tokens and the token's band embedding, so a band can be switched off for a given query. This borrows S³AM's reliability-gate idea and changes what conditions it: S³AM gates between sensor modalities for salient objects using visual context only, while our gate is driven by language and applied to camouflage, where foreground contrast is near zero.
The third part is analysis. We record the gate values and the attention from text tokens to frequency tokens, and test whether they track texture semantics. Concretely, we ask whether queries naming spotted or striped animals weight different bands than queries naming smooth ones, and whether swapping the query on a fixed image moves the weights.
Baselines
The naive reference point is the same frozen backbone with only the decoder and low-rank attention updates trained on RGB patches plus text tokens, with no frequency tokens. This is plain spatial parameter-efficient tuning with language. Any gain the proposal shows must first clear it.
The strongest unimodal frequency competitor is EVP (Liu et al.). It is the method that most directly addresses rigidity with explicit high-frequency injection, and its reported 0.843 S-measure on COD10K is the number to match on standard camouflage segmentation. We will re-run it on our backbone with its fixed FFT high-pass split. We will also run an "EVP + text" variant that adds the same text tokens to the sequence, which isolates whether learned, query-gated bands beat a fixed split once language is present. FGSA-Net (Zhang et al.) is the second frequency baseline. Its ring-partitioned FFT attention inside side adapters is learnable band re-weighting without language. It reports a 2.50% S-measure gain and 34.3% MAE reduction on NC4K over UEDG, and the file gives no absolute value.
The strongest referring competitor is MLKG (Cheng et al.), which reports S-measure 0.910 and MAE 0.019 on R2C7K. R2CNet reports weighted F-measure 0.669 on R2C7K, using visual exemplars as the reference. Beating MLKG outright is a high bar because its LLaVA knowledge generation adds a large external model. For fairness we will run our model both with category-name queries only and with MLKG-style generated descriptions as the query text, under the same SAM backbone family and LoRA budget. We count it a win if we match MLKG's S-measure within noise while using no external MLLM at inference, or exceed it when given the same descriptions.
The key ablation removes the text conditioning from the gate. Frequency tokens still enter the sequence, but band weights become learned constants. This separates "explicit spectral tokens help" from "language-selected spectral tokens help," and the second is our actual claim.
All methods get the same training data, epochs, input resolution, and a fixed hyperparameter search of equal size, and we report means over three seeds.
Why not just…?
Why not just add a text encoder to EVP and fuse at the decoder? This is the most serious threat. If late fusion of language with an EVP-style frozen ViT matches our results, the single-stream design is unnecessary. OneR (Jang et al.) and the VisualBERT analysis (Li et al.) show that early fusion lets words reshape visual features layer by layer, but that evidence comes from vision-language understanding, not segmentation. We will run exactly this late-fusion variant at matched parameters. If it is within noise of ours on R2C7K, we will report that the band-selection benefit does not need early fusion and will recast the contribution as the gate alone.
Why not just unfreeze more of the backbone or use larger LoRA ranks? If rigidity is simply a capacity limit, more trainable parameters should close the gap without any transform. EVP reports that its explicit prompts outperform full fine-tuning on low-level segmentation, which argues against this. Our layer-wise probe tests it directly: if band energy is linearly decodable from frozen features, higher-rank LoRA may indeed suffice. We include a LoRA-rank sweep up to the parameter count of our full method to check.
Why not just prompt a multimodal LLM zero-shot? MMCPF (Chain of Visual Perception) reports F-beta 0.680 on CAMO zero-shot and only 0.196 on MoCA-Mask, and the file notes a large error gap to supervised methods. MLKG also documents that MLLMs miss suppressed edge gradients. We include MMCPF numbers as a reference rather than re-implementing it.
Proposed Experiments
The main benchmark is R2C7K for referring camouflage, with category names converted to text queries and MLKG-style descriptions as a second query condition. We use COD10K, CAMO, NC4K and CHAMELEON for standard camouflage segmentation, with a fixed generic query such as "the hidden animal", so the comparison to EVP and FGSA-Net is direct. OVCamo tests whether band selection transfers to unseen category names. Metrics are S-measure, weighted F-measure and MAE, plus training memory and throughput. The memory and throughput numbers make the cost of the 256 extra tokens visible.
Three checks would settle the claim. The first is query sensitivity. On R2C7K images containing two candidate objects, swapping the query should change both the predicted mask and the gate pattern, and a frozen-gate ablation should lose more accuracy on texture-heavy categories than on smooth ones. The second is signal dependence. Replacing the frequency tokens with ones computed from a low-pass-filtered image should erase the gain, which shows the model uses genuine high-frequency evidence rather than extra tokens as generic capacity. The third is cross-dataset generalization: training on COD10K and testing on NC4K and CAMO without tuning, where the file notes supervised camouflage models struggle. If language-selected bands are a real mechanism rather than dataset fitting, the gap to EVP should widen out of domain rather than shrink.
References
- Explicit Visual Prompting for Low-Level Structure Segmentations
- Large Model Based Referring Camouflaged Object Detection
- Bridging the Pathology Domain Gap: Efficiently Adapting CLIP for Pathology Image Analysis with Limited Labeled Data
- DCPT: Darkness Clue-Prompted Tracking in Nighttime UAVs
- Frequency-Guided Spatial Adaptation for Camouflaged Object Detection
- S$^3$AM: A Single-Stream SAM with Reliability-Calibrated Frequency Adapter for Multi-modal Salient Object Detection
- Referring Camouflaged Object Detection
- Unifying Vision-Language Representation Space with Single-Tower Transformer
- What Does BERT with Vision Look At?
- Chain of Visual Perception: Harnessing Multimodal Large Language Models for Zero-shot Camouflaged Object Detection