Why Keyword Search Is Not Enough
BM25 and TF-IDF match words, not meaning. When users say "affordable" and documents say "budget," keyword search sees zero overlap. This vocabulary mismatch problem is the fundamental motivation for everything in Chapter 6.
Keyword search is the backbone of information retrieval, but it operates on lexical matching: a document only matches if it contains the exact terms in the query. This creates a deep, structural problem for real-world search where users and documents express the same concept in wildly different words.
~20%
Any two people choose the same word for a concept only 20% of the time (Furnas et al., 1987).
5
Synonym blindness, polysemy, conceptual queries, cross-domain gaps, short queries.
Hybrid
Neither keyword nor semantic alone — production systems always combine both approaches.
1. The Vocabulary Mismatch Problem
George Furnas and colleagues at Bell Labs conducted a landmark study in 1987 that quantified what every search engineer eventually discovers: people are remarkably inconsistent in the words they choose to describe the same concept. When asked to name a common object or action, two people independently chose the same word only about 20% of the time. This means that for any given query-document pair describing the same concept, there's an 80% chance the user's query term simply doesn't appear in the relevant document.
For BM25 and TF-IDF, which score relevance based on exact term overlap between query and document, this is catastrophic. A document about "affordable smartphones" scores zero for the query "budget mobile phones" because there's no term overlap — despite perfect semantic relevance. The search system doesn't return bad results; it returns no results for queries where the vocabulary happens to diverge.
This isn't a bug to be fixed with clever engineering — it's a fundamental limitation of the bag-of-words assumption that underlies all keyword-based retrieval. As long as scoring depends on matching character sequences, any vocabulary gap between user and author creates a blind spot that no amount of BM25 parameter tuning can resolve.
The vocabulary mismatch problem is pervasive, not exceptional. It affects every domain, every language, and every user population. The 20% agreement rate from Furnas et al. has been replicated across dozens of studies. Any search system that depends solely on lexical matching is fundamentally limited to serving at most 20% of user information needs optimally.
2. Real-World Impact by Domain
The vocabulary mismatch isn't an abstract academic concern — it costs real money, wastes human time, and in some domains, risks human safety. Understanding the impact by domain reveals why different industries are investing millions in semantic search.
E-Commerce
A user searching "sneakers" won't find products listed as "running shoes" or "trainers." The store has exactly what the customer wants, but keyword search returns zero results — and the customer leaves. At scale, this vocabulary gap translates directly to lost revenue. Amazon estimates that every 100ms of latency costs 1% of sales; invisible inventory due to vocabulary mismatch costs far more.
Healthcare
A patient searching "heart attack symptoms" might miss a critical article about "myocardial infarction indicators." The information isn't just useful — it could be life-saving — but the lexical barrier prevents discovery. Medical terminology creates an extreme version of the cross-domain vocabulary gap.
Legal
Attorneys looking for cases about "breach of contract" need to also find "contractual violation," "default on agreement," and "failure to perform obligations." Missing any of these synonymous formulations means missing potentially critical precedent that could change case outcomes.
Enterprise Knowledge
Employees searching "how to set up VPN" might not find the IT article titled "Remote Access Network Configuration Guide." The knowledge exists, people need it, but vocabulary mismatch makes it invisible. This leads to duplicate documentation, support ticket escalation, and knowledge silos.
3. Five Failure Modes of Keyword Search
The vocabulary mismatch manifests in five distinct patterns, each representing a different type of linguistic gap that keyword search cannot bridge. Understanding these patterns reveals why semantic search exists and what specific problems it solves.
1. Synonym Blindness
The most obvious failure: different words for the same concept produce zero match. "automobile" vs. "car" vs. "vehicle" vs. "motor car" — these are all the same thing, but BM25 treats them as completely unrelated terms. A document about "automobile maintenance" is invisible to a query for "car repair."
Traditional mitigation is synonym dictionaries, but these are fundamentally limited. They require manual curation by domain experts, must be updated constantly as language evolves, can introduce noise (not all synonyms are appropriate in all contexts), and never achieve complete coverage. A synonym dictionary might define "car ↔ automobile," but it won't capture "whip" (slang), "ride," or industry terms like "unit" (car dealerships).
2. Polysemy (Same Word, Different Meaning)
The inverse problem: a single word has multiple unrelated meanings, and keyword search cannot distinguish between them. "Bank" could mean a financial institution, a river bank, a blood bank, or a data bank. "Apple" could be a fruit or a technology company. "Python" could be a snake, a programming language, or a Monty Python reference.
When a user searches for "python crash," are they looking for a snake attack video or a Python programming error? BM25 has no way to know — it matches documents containing both "python" and "crash" regardless of meaning. This is not a problem synonym dictionaries can solve. It requires understanding the context around words, which is exactly what embedding models provide.
3. Conceptual and Intent-Based Queries
Modern users increasingly express their search needs as questions or concepts rather than keyword fragments: "things to do when bored," "how does the economy affect housing prices," "why does my car make a grinding noise when I brake." BM25 can only work with the individual terms — documents about "entertainment ideas" or "hobby suggestions" won't match because they don't contain the word "bored."
This is becoming increasingly critical as search interfaces evolve. Voice search, conversational AI, and natural language interfaces all produce full sentences, not keyword fragments. A search system built only for "laptop reviews 2024" will fail when users ask "what laptop should I buy for college?"
4. Cross-Domain and Cross-Register Gaps
Technical communities develop specialized vocabulary that doesn't overlap with everyday language. A software engineer's "deploy to production" is a manager's "launch the update." A doctor's "idiopathic etiology" is a patient's "they don't know what's causing it." A lawyer's "tortious interference" is a layperson's "they messed up my business deal."
When the searcher and the document author come from different professional or cultural backgrounds, keyword matching breaks down completely. The semantic meaning is identical, but the surface-level text shares no terms.
5. The Short Query Problem
Most real-world search queries are surprisingly short — typically 2-3 words. This gives BM25 very little signal to work with. "python error" — which error? In what context? What version? "best restaurant" — best by what criteria? What cuisine? What location?
With so few terms, keyword search returns an overwhelming number of partially relevant results. The user must dig through pages of results, mentally filtering based on context that the search engine couldn't understand. Embedding models address this by mapping even short queries into a rich vector space where proximity to documents captures latent meaning beyond the literal words.
4. Traditional Attempts to Fix This
Search engineers have developed several techniques to bridge the vocabulary gap within keyword-based systems. While each helps incrementally, none solves the fundamental problem. Understanding their limitations explains why the industry moved to embedding-based approaches.
Stemming and Lemmatization reduce words to their root form: "running" → "run," "better" → "good." This helps with morphological variations but doesn't cross word boundaries. "Automobile" will never stem to "car." Synonym Dictionaries map related words but require human curation, can't scale, get stale, and introduce false synonyms — "light" means "not heavy" in "light bag" but "illumination" in "light source."
Query expansion automatically adds related terms but dramatically hurts precision because the expanded terms are often noisy. Expanding "java" to include "coffee" will pollute programming search results. Pseudo-relevance feedback takes top results from the initial query, extracts common terms, and re-runs — but amplifies any errors in the initial results. If the first page of results is off-topic, PRF pushes results further off-topic.
| Technique | What It Helps | What It Can't Fix |
|---|---|---|
| Stemming | "running" ↔ "run" | "car" ↔ "automobile" |
| Synonyms | "car" ↔ "automobile" | Context-dependent meanings |
| Query expansion | Related terms | Precision (adds noise) |
| N-grams | Multi-word phrases | Conceptual matching |
| PRF | Domain vocabulary | Initial bad results cascade |
5. The Semantic Search Promise
Semantic search solves the vocabulary mismatch by representing both queries and documents as dense vectors in a shared embedding space where geometric proximity encodes semantic similarity. The core insight is revolutionary: instead of comparing words, compare meanings. An embedding model learns that "cheap flights" and "budget airline tickets" express the same intent, even though they share zero words.
This works because embedding models are trained on billions of text examples, learning that words appearing in similar contexts have similar meanings. The model doesn't need a synonym dictionary — it has learned a continuous, high-dimensional map of language where meaning, not spelling, determines location.
6. Why Not Replace Keyword Search Entirely?
If semantic search is so powerful, why not replace keyword search entirely? Because semantic search has its own failure modes that are the mirror image of keyword search's weaknesses. Understanding these is critical for making the right architectural decisions.
🎯 Exact Match Failure
Searching for "SKU-2847-B" returns similar-looking SKUs because embeddings treat all alphanumeric codes as roughly equivalent. Keyword search finds the exact string instantly. Product IDs, case numbers, error codes — all require exact matching.
📉 Information Loss
Embeddings compress an entire document into ~768 floats. Specific numbers, dates, measurements, and exact phrases are lost. "What is the horsepower of a 2024 Camry?" requires exact facts, not semantic similarity.
🚫 Negation Blindness
"Hotels with pool" and "hotels without pool" produce nearly identical embeddings (cosine ≈ 0.95). The logical negation "without" barely changes the vector because bi-encoders are trained for topical similarity, not logical structure.
💰 Computational Cost
Generating embeddings requires neural network inference. Comparing billions of vectors requires specialized indexes and significant RAM. All of this costs significantly more than inverted indexes. At billion-scale, vector infrastructure can cost $15-30K/month.
Hybrid search (Chapter 6.7): combine keyword precision with semantic understanding. BM25 handles exact IDs, negation, and structured queries. Vectors handle vocabulary mismatch, conceptual queries, and intent matching. Together they provide robust retrieval for every query type.
Key Takeaways
Vocabulary Mismatch Is Fundamental
Users and authors describe the same concept with different words. Furnas et al. (1987) showed users agree on the same term only 20% of the time — BM25 finds zero matches for the other 80%. This isn't an edge case; it's the norm.
Five Distinct Failure Modes
Synonym blindness, polysemy confusion, conceptual queries, cross-domain vocabulary gaps, and short queries — each represents a structurally different way lexical matching fails, and no single fix addresses all of them.
Traditional Fixes Are Band-Aids
Stemming, synonym dictionaries, query expansion, n-grams, and pseudo-relevance feedback each help at the margins but introduce new problems. Synonym dictionaries require constant curation and can't handle context-dependent meanings.
Hybrid Search Is the Production Answer
Neither keyword nor semantic search alone is sufficient. Keyword search excels at exact IDs and structured data; semantic search excels at conceptual queries. Modern systems combine both for robust retrieval across all query types.