Most modern Natural Language Processing (NLP) models treat semantic similarity as a geometric problem: they map texts into a shared vector space and measure the distance or angle between them. This approach works reliably when comparing texts of similar scale—such as balancing two short sentences against one another. However, when models are tasked with comparing texts of vastly different lengths—such as a concise summary against a full-length article, or a brief definition against a descriptive paragraph—this geometric assumption breaks down.
This breakdown is caused by length asymmetry. When texts vary significantly in scale, standard similarity metrics suffer from systematic mathematical and cognitive distortions that prevent them from recognizing true semantic equivalence.
The Dual Distortions: Dilution and the Wordiness Reward
Length asymmetry distorts automated similarity metrics in two opposing ways, depending on whether the underlying algorithm relies on dense vector representations or sparse token overlaps.
The Dilution Effect
In dense embedding spaces or structural comparison frameworks, comparing a concise text to a highly verbose one artificially deflates the similarity score. As demonstrated by Daniel Bär et al. in Text Reuse Detection using a Composition of Text Similarity Measures, when a text is significantly expanded with new details, the proportion of core reused content is diluted.
This dilution creates a severe distributional misalignment, a phenomenon explored by Austin Xu et al. in SMILE: A Composite Lexical-Semantic Metric for Question-Answering Evaluation. Because the rich context of a long document introduces a wider array of vocabulary and syntactic structures, its vector representation diverges sharply from a concise counterpart, as shown by Xuchao Zhang et al. in Unsupervised Concept Representation Learning for Length-Varying Text Similarity. Consequently, standard metrics like cosine similarity unfairly penalize length imbalances, underestimating semantic equivalence in tasks ranging from short-text definition matching to web page clustering and name disambiguation (Derek F. Wong et al. in A Template Based Hybrid Model for Chinese Personal Name Disambiguation; Tiantian Zhu et al. in ECNUCS: Measuring Short Text Semantic Equivalence Using Multiple Similarity Measurements; Agustín D. Delgado et al. in A Data Driven Approach for Person Name Disambiguation in Web Search Results).
The Wordiness Reward
Conversely, high-dimensional vector models and lexical overlap metrics frequently suffer from a mathematical bias that rewards sheer verbosity. Longer text sequences inherently possess a higher statistical probability of random token or concept overlap.
The authors of Listening Comprehension over Argumentative Content observed that semantic similarity models used for argument detection frequently flag longer sentences as matches simply due to this statistical inflation. Similarly, Dan Iter et al. in Automatic Detection of Incoherent Speech for Diagnosing Schizophrenia proved that traditional semantic models mistake wordiness for logical flow, incorrectly scoring verbose but disorganized speech as highly coherent. This mathematical artifact allows models to "cheat" by relying on superficial length features rather than genuine semantic alignment, a vulnerability highlighted by Haochen Tan et al. in A Sentence is Worth 128 Pseudo Tokens: A Semantic-Aware Contrastive Learning Framework for Sentence Embeddings.
[Short Text] --> Concentrated Semantic Signal
VS.
[Long Text] --> Diluted Signal + High Statistical Overlap Probability
The Cognitive Disconnect
The root of this engineering challenge lies in a fundamental disconnect between machine math and human cognition. Human perception of textual similarity does not scale linearly with text length.
As established by Nikos Malandrakis et al. in DeepPurple: Estimating Sentence Semantic Similarity using N-gram Regression Models and Web Snippets, human evaluators alter their cognitive strategies based on the scale of the text. When comparing short sentences, humans focus primarily on lexical and surface-level matches. However, when evaluating longer texts, the human brain shifts to higher-level cognitive processing, weighing abstract concepts and global themes while filtering out minor descriptive details.
Standard similarity models that apply a uniform mathematical approach across all text lengths fail to capture these changing cognitive weights, as argued by Nikolaos Malandrakis et al. in DeepPurple: Lexical, String and Affective Feature Fusion for Sentence-Level Semantic Similarity Estimation. Without explicit mechanisms to account for this non-linear relationship, automated metrics diverge significantly from human judgments of semantic equivalence (Alberto Barrón-Cedeño et al. in UPC-CORE: What Can Machine Translation Evaluation Metrics and Wikipedia Do for Estimating Semantic Textual Similarity?; Hoang-Quoc Nguyen-Son et al. in Paraphrase Detection Based on Identical Phrase and Similar Word Matching).
Bridging the Structural Gap
To resolve length-induced bias, recent architectures abandon the assumption that text must be compared at a single, uniform structural level. Instead, they employ two primary strategies: cross-granularity alignment and decoupled concept spaces.
Cross-Granularity Alignment
Rather than forcing a short query and a long document into the same vector representation, cross-granularity models break the texts down and map connections across different linguistic levels.
Query (Phrase Level) -----> [ Word-Level Alignment ] -----> Supporting Text
-----> [ Sentence-Level Alignment ] -----> (Paragraph)
By calculating similarities across multiple granularities simultaneously—such as aligning a single phrase to a word or an entire sentence—the system can identify strong semantic anchors that span different structural scales. This multi-level alignment strategy allows models to bypass the normalization traps of single-vector comparisons, as demonstrated by Steven Bethard et al. in Alignment over Heterogeneous Embeddings for Question Answering and Peyman Passban et al. in Improving Phrase-Based SMT Using Cross-Granularity Embedding Similarity.
Decoupled and Concept-Driven Spaces
An alternative strategy isolates the underlying semantic content from the text's physical length. Frameworks like CGNet, developed by Xuchao Zhang et al., learn corpus-level "concepts" and use an augmented Wasserstein distance (optimal transport) to match texts, allowing them to recognize semantic equivalence even when vocabulary and context density vary drastically.
Similarly, Xixi Zhou et al. in Matching Varying-Length Texts via Topic-Informed and Decoupled Sentence Embeddings introduced methods to decouple domain-specific topic modeling from dense sentence embeddings. By separating the thematic essence of a document from its structural verbosity, these frameworks establish a stable, length-insensitive embedding space that prevents the truncation and dilution issues that plague traditional transformer architectures.
Key Foundational Work
DeepPurple: Estimating Sentence Semantic Similarity using N-gram Regression Models and Web Snippets (2012) — contributed 2 concept elements to this direction.
A Template Based Hybrid Model for Chinese Personal Name Disambiguation (2012) — contributed 1 concept elements to this direction.
Automatic generation of inter-passage links based on semantic similarity (2010) — contributed 1 concept elements to this direction.