If you ask a human a question, rephrase it slightly, or introduce a minor typo, you expect the core meaning of their answer to remain unchanged. If you ask a large language model (LLM) to perform the same task, however, this expectation frequently falls apart.
Generative models exhibit a profound fragility: minor, semantically insignificant variations in input text—such as switching synonyms, altering formatting, or changing word order—can trigger radically different, and sometimes contradictory, outputs. This phenomenon, often framed as input sensitivity or prompt multiplicity, presents a major roadblock for deploying these models in high-stakes, safety-critical domains like medicine, finance, and infrastructure management.
The Illusion of Semantic Stability
At their core, autoregressive transformers map surface-level tokens to probabilistic distributions. Because they lack an explicit, invariant representation of underlying concepts, they are highly sensitive to the exact geometry of the input prompt. As Tuan et al. observe in Generative Dictionary: Improving Language Learner Understanding with Contextual Definitions, this instability makes ensuring reliable text generation incredibly difficult, even for straightforward tasks like generating word definitions.
This vulnerability is not unique to text-to-text models. In text-to-image synthesis, Yin et al. demonstrate in Semantics Disentangling for Text-To-Image Generation that standard generative models frequently produce visually distinct or degraded images when given synonymous descriptions of the same scene. The models struggle to separate "linguistic fluff" from the core semantic identity of the prompt.
When models are deployed in isolation for critical tasks, this sensitivity turns into a liability. For example, Cercola et al. evaluated LLMs for highway traffic incident management in Automating the loop in traffic incident management on highway and found that minor phrasing changes drastically altered the models' recommended emergency actions, highlighting a dangerous lack of decision consistency.
The Compound Fragility of Retrieval-Augmented Generation
To combat hallucinations and anchor models in factual truth, system architects often turn to Retrieval-Augmented Generation (RAG). RAG systems fetch relevant documents from an external database to ground the model's response. However, adding a retrieval step does not solve input sensitivity; instead, it introduces a compound vulnerability where both the retriever and the generator act as distinct bottlenecks.
[User Query] ──(Variations/Typos)──► [Retriever] ──► Fetches Non-Overlapping Docs
│
▼
[Generator] ──► Produces Inconsistent Answers
As Ganesh et al. argue in Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicity, while RAG can improve overall factual correctness, it frequently introduces new forms of inconsistency, shifting errors toward randomness rather than resolving them. This compound fragility operates on two levels:
- The Retriever Bottleneck: Small changes in a query's keywords or syntax can cause dense retrievers to surface entirely different, non-overlapping document sets. As shown by Ho et al. in Rewrite-to-Rank: Optimizing Ad Visibility via Retrieval-Aware Text Rewriting, minor variations in phrasing or formatting dictate whether a piece of information is surfaced or completely buried by automated pipelines.
- The Generator Bottleneck: Even if the retriever manages to pull the correct documents, the generator remains highly sensitive to the phrasing of the user's prompt. In fact, Peng et al. demonstrate in Harmonizing Diverse Models: A Layer-wise Merging Strategy for Consistent Generation that the generator component is often the primary source of response inconsistency in RAG systems, proving significantly more sensitive to minor query variations than the retriever itself.
This dual-component sensitivity means that a simple typo can degrade performance across the entire pipeline. In Investigating the Robustness of Retrieval-Augmented Generation at the Query Level, Perçin et al. systematically confirmed that query-level perturbations cause significant performance drops, threatening the reliability of these architectures in production.
This challenge isn't entirely new. As Lease et al. point out in Preaching to the ChoIR: Lessons IR Should Share with AI, the modern AI community is currently rediscovering a hard-won lesson that Information Retrieval (IR) researchers mastered decades ago: user query formulation is inherently variable, and building stable user interfaces requires systems that are robust to this variation.
Strategies for Enforcing Consistency
To build systems that users can trust, researchers are developing methods to force models to find "semantic common ground," ensuring uniform outputs across paraphrased inputs.
Reinforcement Learning via Group Rewards
One prominent approach is to explicitly reward models for consistency during alignment. In Improving Consistency in Retrieval-Augmented Systems with Group Similarity Rewards (and expanded in their later work, Improving Consistency in Retrieval Augmented Systems with Group Similarity Reward), Hamman et al. introduce Con-RAG.
Instead of evaluating a model on a single query, Con-RAG feeds a group of semantically equivalent, paraphrased queries to the system simultaneously. It uses a reinforcement learning algorithm called PS-GRPO to optimize a "group similarity reward," penalizing the model if its answers diverge across the variations. This trains the generator to synthesize consistent answers even when the retriever provides inconsistent source evidence.
Layer-Wise Model Merging
Rather than relying solely on reinforcement learning, Peng et al. propose a structural solution in Harmonizing Diverse Models: A Layer-wise Merging Strategy for Consistent Generation. They train separate, specialized models on different subsets of query variations and then combine them using a novel layer-wise model merging strategy. This smooths out the erratic activation spaces of individual models, yielding a final architecture that maintains high accuracy while resisting query-level perturbations.
Disentangled Architectures and External Grounding
In domains outside pure text generation, architectural constraints are used to isolate semantic meaning from surface phrasing. For text-to-image tasks, the SD-GAN framework introduced by Yin et al. utilizes a Siamese structure and a specialized "Semantic-Conditioned Batch Normalization" (SCBN) technique to explicitly separate the core concept from linguistic variations, ensuring visual consistency.
Alternatively, researchers mitigate sensitivity by anchoring the generative model to rigid, non-generative reasoning tools. For medical diagnosis, Ju et al. introduce Prediction-Augmented Generation (PAG) in Prediction-Augmented Generation for Automatic Diagnosis Tasks. By forcing the LLM to base its reasoning on the statistical outputs of specialized predictive models, PAG prevents minor descriptive variations from throwing off the diagnosis.
Similarly, the hybrid approach explored by Cercola et al. pairs LLMs with classical mathematical optimization (like Markov Decision Processes) to ensure that even if the prompt phrasing fluctuates, the final safety-critical decisions remain mathematically grounded and stable.
Key Foundational Work
Semantics Disentangling for Text-To-Image Generation (2019) — contributed 1 concept elements to this direction.
Generative Dictionary: Improving Language Learner Understanding with Contextual Definitions (2024) — contributed 1 concept elements to this direction.