Semantic Search Beyond Keyword Matching

55 concepts46 papers2 proposals

For decades, information retrieval relied on a simple assumption: if a query and a document share the exact same words, they must be relevant to each other. This lexical matching paradigm powers traditional search engines, but it breaks down when language becomes nuanced. Human intent is messy—we use synonyms, speak with ambiguous phrasing, or describe concepts without ever naming them directly.

To bridge this gap, a long-standing research direction has focused on meaning-based information retrieval. Instead of treating text as a sequence of character strings, this approach maps language to underlying concepts, intents, and semantic structures. By decoupling the surface form of language from its core meaning, systems can understand what a user intends to find, rather than just what they typed.

The Core Intuition: From Strings to Concepts

To understand why meaning-based retrieval works, consider how humans process a query. If someone says, "The letters are too small," they are not looking for documents containing the words "letters" or "small." They are expressing an intent to change their font size.

Lexical search fails here because it looks for exact string overlaps. Meaning-based search, however, translates both queries and documents into a shared conceptual space.

Lexical Search:
"letters too small" ───(Exact Match)───> Documents with ["letters", "small"]

Meaning-Based Search:
"letters too small" ───(Abstraction)───> [Concept: Accessibility / Font Size] ───(Retrieval)───> Relevant Settings/Docs

Early foundations for this paradigm focused on organizing information explicitly around conceptual categories rather than keywords, as seen in foundational work like the framework described in Organization and Retrieval in a Conceptual Memory. Decades ago, H. Zimmermann demonstrated that treating natural language problem descriptions as conceptual indices allowed non-expert users to navigate complex documentation systems far more intuitively than rigid keyword queries allowed.

Explicit Conceptual Mapping: Ontologies and Graphs

One major mechanism for capturing meaning involves mapping text to structured, human-engineered knowledge bases or ontologies.

Instead of guessing the meaning of a word in isolation, systems can use the surrounding context to pin down the exact concept being discussed. For instance, Juan Manuel Madrid Molina developed a hybrid approach in KeyConcept: A Conceptual Search Engine that automatically infers semantic concepts from descriptive text and maps them to a structured ontology. This allows the engine to filter results based on the inferred category rather than raw keyword overlap.

Similarly, text can be transformed into conceptual graphs that explicitly map relationships between entities. Zhitao Guan et al. leveraged this in Achieving Semantic Secure Search in Cloud Supported Information-Centric Internet of Things, demonstrating that structured semantic relationships can even be utilized to perform complex, intent-based searches over encrypted cloud data.

Resolving Ambiguity via Semantic Context

A fundamental flaw of keyword search is polysemy—the fact that a single word can have multiple meanings (e.g., "bank" as a financial institution versus a riverbank). Meaning-based retrieval solves this through Word Sense Disambiguation (WSD).

By integrating automatic WSD into the indexing process, P.G. Ramasubramanian et al. showed in An Intelligent Information Retrieval System using Automatic Word Sense Disambiguation that treating words as distinct concepts rather than character strings dramatically increases search precision. This transition from shallow text tagging to deep grounding in a structured repository of concepts was further formalized by German Rigau et al. in MEANING: a Roadmap to Knowledge Technologies, proving essential for advanced tasks like cross-lingual retrieval and semantic question answering.

In specialized domains, this semantic comprehension allows systems to simulate human-like reasoning. For example, Guangquan Xu et al. developed a programmer-oriented Q&A system in POQAS-S: a Novel Programmer-Oriented Online Question Answering System With Semantic Comprehension that interprets the underlying context of technical queries, achieving significantly higher accuracy and speed than traditional keyword tools.

The Modern Shift: Dense Semantic Spaces

While early systems relied on explicit graphs and ontologies, modern meaning-based retrieval increasingly uses deep neural networks to project text into continuous, high-dimensional vector spaces (dense embeddings). In these spaces, geometric proximity reflects semantic similarity.

Keyword Space (Orthogonal):
"Font"     ──────> [1, 0, 0]
"Letters"  ──────> [0, 1, 0]  (No relationship captured)

Dense Semantic Space (Vector Proximity):
"Font"     ──────> [0.82, 0.11, 0.45]
"Letters"  ──────> [0.80, 0.14, 0.42]  (Close together in space)

This dense retrieval approach excels at handling real-world data noise. As Chris Samarinas et al. demonstrated in Improving Evidence Retrieval for Automated Explainable Fact-Checking, dense semantic retrieval models significantly outperform keyword-based methods when identifying evidence across noisy datasets where the language varies wildly from the target claim.

A striking application of this intuition is found in mobile interface navigation. Joon-Young Kim et al. trained a neural retrieval model using contrastive learning in Intuitive Access to Smartphone Settings Using Relevance Model Trained by Contrastive Learning. Their system allows users to find settings using natural, descriptive language (like the "letters are too small" example), increasing the success rate of finding the correct setting by over four times compared to traditional lexical matching.

Evidence from Real-World Behavior

The necessity of meaning-based retrieval is underscored by how humans actually interact with search systems:

Key Foundational Work

International Conference on Computational Linguistics COLING 1969: Preprint No. 4 (1969) — contributed 2 concept elements to this direction.
Natürlichsprachige Problembeschreibung als ein Verfahren für den Bürgernahen Zugang zu Dokumentationssystemen (1980) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_meaning_based_information_retrieval_56305fda,
  title = {meaning-based information retrieval},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/meaning-based-information-retrieval/rdr_56305fda470445b8a1bd4d9b4e8e81d8},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_meaning_based_information_retrieval_56305fda_v1,
  title = {meaning-based information retrieval},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/meaning-based-information-retrieval/rdr_56305fda470445b8a1bd4d9b4e8e81d8/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake