Text preprocessing is usually treated as a yes-or-no decision made once before training: either every word gets its spelling normalized, or none does. The evidence says this is the wrong unit of choice. On the same Arabic social-media dataset, letter normalization is what a linear sentiment classifier needs, while raw, uncleaned text gives the best sarcasm detection, because elongated letters and punctuation runs carry the irony (Preprocessing Solutions for Detection of Sarcasm and Sentiment for Arabic). In dialects without a fixed spelling, the same tension shows up inside a single sentence. Some variant spellings are noise that splits one word into many unrelated tokens. Others are the very signal of dialect identity that standard-language models erase (EDRAC).
We propose replacing the on/off pipeline with a small learned layer that blends each word's raw and normalized representations by a continuous amount. The blend is governed by the equations of admittance control, the robotics technique that lets a stiff arm yield smoothly to contact forces instead of fighting them. The normalized form plays the role of the robot's planned path. Evidence from the raw spelling plays the role of the external force that pushes the representation off that path. A learned stiffness sets how hard the model pulls back toward the standard form, and damping keeps the blend from jumping erratically from word to word. The payoff is twofold. One architecture should handle tasks that want opposite amounts of cleanup. The learned stiffness also becomes a readable, continuous measurement of how much normalization a task actually wants.
Core Claim
Our claim is that a per-word, sequence-smoothed interpolation between raw and normalized representations can match or beat the best task-specific binary pipeline on every task at once, and that its smoothing across neighboring words matters. Concretely, let $e^{\text{norm}}_t$ and $e^{\text{raw}}_t$ be the word-level embeddings of word $t$ under normalized and raw input, obtained by pooling subword vectors over word spans. We align the two views at the word level, which works because the normalizations we use, such as letter unification and elongation stripping, operate within words. The representation passed upward is $h_t = e^{\text{norm}}_t + \Delta_t$, where the deviation $\Delta_t$ follows a discretized mass-spring-damper recurrence along word position:
$$M(\Delta_t - 2\Delta_{t-1} + \Delta_{t-2}) + D(\Delta_t - \Delta_{t-1}) + K\Delta_t = F_t, \qquad F_t = W\,(e^{\text{raw}}_t - e^{\text{norm}}_t).$$
Here $M$, $D$, $K$ are learned diagonal matrices (virtual inertia, damping, stiffness), and $W$ is a learned map that turns the raw-versus-normalized discrepancy into a "force." With $M = D = 0$ the layer reduces to $\Delta_t = K^{-1}F_t$. That is a memoryless per-word gate, the textual analogue of the proportional controller that Compliance control based on PSO algorithm to improve the feeling during physical human–robot interaction found produces jerky transitions, which second-order dynamics smooth out. We parameterize the recurrence by natural frequency and damping ratio, constrained so its characteristic roots stay inside the unit circle and the filter cannot diverge. The bet behind the second-order terms is that orthographic deviation is clustered: dialectal stretches, code-switched spans, and emotive elongation tend to come in runs, so a word's neighbors are evidence about how much to trust its raw form. This bet can fail, and the experiments are built to show whether it does.
Relation to Prior Work
The motivating evidence comes from work showing that preprocessing value depends on the task. Story Link Detection and New Event Detection are Asymmetric shows that precision-oriented linking needs exact variants while recall-oriented event detection wants aggressive normalization. An In-depth Analysis of the Effect of Text Normalization in Social Media finds that edits that help speech synthesis hurt parsing. Lichouri et al. reach the same conclusion within one Arabic dataset. All three treat preprocessing as a discrete choice made per task. We make it continuous and per word, and we let one model hold both regimes. Exploring the Relationship Between Algorithm Performance, Vocabulary, and Run-Time in Text Classification maps accuracy-latency trade-offs for static English bag-of-words pipelines and notes that chained steps interact unpredictably. We borrow its stance that preprocessing is a measurable trade-off, but we move it to subword-tokenized encoders and nonstandard orthography.
Two lines of dialect work set constraints on the design. A Conventional Orthography for Algerian Arabic supplies expert rules mapping dialect to a conventional script. Dialect Representation Learning with Neural Dialect-to-Standard Normalization learns dialect-to-standard rewriting with sequence-to-sequence models. Both commit hard to the standard output. We reuse such normalizers only as the "planned path," and we never discard the raw view. Add Noise, Tasks, or Layers? shows that random character noise and layer swapping make Norwegian dialect models more robust, and that end-to-end training on about 2.9k dialect examples overfits badly. That overfitting finding constrains us: the compliant layer adds only a few diagonal parameters per dimension plus $W$, and we test whether damping also acts as a regularizer in this small-data regime.
From robotics we take only the mechanism. FAVLA and Navigation beyond Wayfinding modulate compliance online, switching between rigid tracking and yielding. Their compliance lives in physical end-effector space. Nothing in the file applies these dynamics to representations, and we make no claim that the physics carries meaning in text beyond the recurrence it provides.
Research Program
We will build the compliant normalization layer as a drop-in module between the embedding layer and the first transformer block of a pretrained multilingual encoder. mDeBERTa, used by Blaschke et al., is the shared backbone so results are comparable across languages. For Arabic, the normalized view applies standard letter normalization and elongation stripping. A second, heavier variant uses CODA-style conventionalization where rules exist. For Norwegian, the normalized view comes from a dialect-to-standard normalizer in the style of the NDC work.
The analysis has three parts. The first is accuracy against binary pipelines. The second is the learned stiffness per task. We predict sarcasm detection learns lower stiffness than sentiment on the same ArSarcasm-v2 inputs, and that city-level dialect identification learns the lowest of all, because the deviations are the label. The third is per-word deviation profiles, used to check whether the model yields on linguistically sensible spans, such as elongation and dialect-specific morphemes like the Algerian "N of Number" construct, rather than on arbitrary tokens. As a secondary measurement we report inference cost, since the layer requires encoding two views, in the spirit of Fearn et al.
Baselines
The naive reference points are the same backbone fine-tuned on raw text only and on normalized text only. Together they bracket what any blending method must beat. The strongest existing practice is choosing the best binary pipeline per task on the development set, as Lichouri et al. did. We reimplement their surface-cleaning toggles and all their combinations on mDeBERTa, select per task on dev, and give this baseline the same hyperparameter-search budget as our method. On ArSarcasm-v2 the file reports Lichouri et al.'s test F1-sarcastic of 33.71% and F1-PN of 57.87%, about 7–8 points below the average participant. It reports no result for the strongest ArSarcasm-v2 system, so the bar is our own tuned pipeline baseline, not that number.
For Norwegian, the strongest reported system is MaiNLP with layer swapping and noise, at 97.6% intent accuracy and 85.6% slot F1 (Add Noise, Tasks, or Layers?). We rerun their 20% character-noise recipe on our backbone. We also add our layer on top of it, since the two are complementary rather than exclusive. For MADAR city-level identification, the reference is ArbDialectID's 67.32% macro F1 with n-gram feature ensembles (ArbDialectID).
The key ablations isolate the ingredients. A two-view concatenation baseline feeds both embeddings to the encoder without any gate. A single scalar learned mixing weight per task tests whether per-word decisions matter. The memoryless per-word gate ($M = D = 0$) tests whether the second-order smoothing matters. Every variant gets identical data, seeds (at least five), epochs, and tuning trials. Beating a baseline means a mean improvement larger than the seed standard deviation, on both sarcasm and sentiment simultaneously for ArSarcasm-v2.
Why not just…?
Why not just use a learned sigmoid gate per word instead of admittance dynamics? This is the real threat. The admittance layer with zero inertia and damping is exactly such a gate. The memoryless ablation decides the question directly. If the gate matches the full layer within seed variance on all three benchmarks, we drop the control-theoretic claim and report the result as "per-word gated normalization." That result would still answer the preprocessing question. The second-order terms earn their place only if they help, most plausibly on code-switched or elongation-heavy subsets where neighboring-word evidence should matter.
Why not just pick the best binary pipeline per task? It is cheap and is our strongest baseline. Its weakness is that it cannot serve a single input that mixes noise-like and signal-like variants, and it cannot serve multiple tasks with one model. Lichouri et al.'s finding that chained steps degrade sarcasm detection suggests the optimum lies between pipelines rather than at any one of them. If per-task selection matches our layer, then per-word decisions add nothing, and we will say so.
Why not just inject character noise into standard training data? Blaschke et al. show it helps intent accuracy but does not give the best slot filling. Noise also teaches invariance to all variation, which is the wrong goal when the variation is the label, as in dialect identification or sarcasm. We test noise alone, our layer alone, and both combined on Norwegian SID. We expect noise to win on intent and our layer to add on slots and on MADAR.
Why not just use a frontier model zero-shot? EDRAC finds GPT-5.4 falls from first to joint fifth on a dialect-aware metric because it drifts to Modern Standard Arabic (EDRAC). That is evidence that scale alone does not preserve dialectal signal. We do not run frontier models on our classification tasks, so we make no claim about their accuracy there.
Decisive Checks
The proposal succeeds if four things hold. First, one compliant model matches or beats the tuned best-pipeline baseline on both ArSarcasm-v2 tasks at once. Second, the full layer beats the memoryless gate on at least the elongation- and code-switch-rich subsets. Third, learned stiffness orders tasks as predicted: dialect identification lowest, then sarcasm, then sentiment, with consistent ordering across seeds. Fourth, on Norwegian SID, adding damping reduces variance across seeds relative to the undamped gate, which would indicate it helps against small-data overfitting. If the stiffness ordering comes out reversed or random, the interpretive claim fails even if accuracy improves. If both the gate and the full layer fail to beat per-task pipeline selection, continuous per-word normalization is not worth its two-view encoding cost on these benchmarks.
References
- Preprocessing Solutions for Detection of Sarcasm and Sentiment for Arabic
- EDRAC: Benchmarking Arabic Dialect Reading Comprehension
- Compliance control based on PSO algorithm to improve the feeling during physical human–robot interaction
- Story Link Detection and New Event Detection are Asymmetric
- An In-depth Analysis of the Effect of Text Normalization in Social Media
- Exploring the Relationship Between Algorithm Performance, Vocabulary, and Run-Time in Text Classification
- A Conventional Orthography for Algerian Arabic
- Dialect Representation Learning with Neural Dialect-to-Standard Normalization
- Add Noise, Tasks, or Layers? MaiNLP at the VarDial 2025 Shared Task on Norwegian Dialectal Slot and Intent Detection
- FAVLA: A Force-Adaptive Fast-Slow VLA model for Contact-Rich Robotic Manipulation
- Navigation beyond Wayfinding: Robots Collaborating with Visually Impaired Users for Environmental Interactions
- ArbDialectID at MADAR Shared Task 1: Language Modelling and Ensemble Learning for Fine Grained Arabic Dialect Identification