Chapter 0.3
What "Good Search" Actually Means
It's not about keyword matching. It's about intent satisfaction.
The Naive Definition (Wrong)
"Good search returns documents that contain the query terms."
This definition fails immediately:
Query: "running shoes"
Result #1: Blog post titled "Why I stopped running shoes are bad for you"
✓ Contains "running" ✓ Contains "shoes" → Completely irrelevant
Query: "cheap laptop"
Result #1: MacBook Pro ($2,499)
✓ It IS a laptop → User said "cheap"!
The Engineering Definition
Good search is about probability of intent satisfaction, not keyword matching. Context includes: who the user is, where they are, what device, time of day, and their history.
The Three Pillars of Good Search
1. Relevance
Does the result match the user's intent?Textual Relevance
Query terms appear in document
Example: "iPhone case" → product contains those words
Semantic Relevance
Meaning matches (even without exact terms)
Example: "mobile phone cover" → matches "iPhone case"
Business Relevance
Available, in budget, shippable
Example: Out of stock items ranked lower
🎯 Real failure:
E-commerce site ranks "iPhone 12" above "iPhone 14" for query "iPhone." Why? iPhone 12 has more reviews (popularity signal). But iPhone 14 is what most people want in 2024.
2. Speed
Did the user get results fast enough?⚡ The trade-off:
Making search "smarter" (more ML, deeper reranking) makes it slower. Good search is the optimal trade-off, not maximal intelligence. A perfectly ranked result that takes 2 seconds is worse than "good enough" in 100ms.
3. Discovery
Did the user find things they didn't know they wanted?Exact Match
"iPhone 15 Pro 256GB Blue" → Exactly that product
Substitute
"iPhone 15" (out of stock) → "iPhone 14 Pro" (available, similar specs)
Serendipity
"running shoes" → "compression socks" (frequently bought together)
✨ The discovery paradox:
Users say they want "exactly what they searched for." But the best search experiences surprise them. Netflix, Spotify, Pinterest win by showing you things you didn't know you wanted.
Good Search in Action: Real Examples
Amazon
- • "running shoes" → filters by Prime eligibility, your size
- • Shows "Customers also bought" for discovery
- • Sponsored results clearly labeled (trust)
- • Sub-100ms response times at massive scale
- • Featured snippets answer questions directly
- • Knowledge panels for entity queries ("Barack Obama")
- • Local results for "coffee near me" (context)
- • "Did you mean" for typos
Netflix
- • Personalized results (your history matters)
- • "Because you watched X" explanations
- • Thumbnails change per user (discovery)
- • Fuzzy matching for actor names
Notion
- • Searches your own content (private)
- • Recent pages weighted higher
- • Cmd+K instant search (speed)
- • Filters by page type, date
Anti-Patterns: What Bad Search Looks Like
| Symptom | Underlying Problem | Fix |
|---|---|---|
| "Zero results" for common queries | Missing synonyms, poor tokenization | Add synonyms, improve query expansion |
| First result is always wrong | Bad ranking features or stale popularity | Add freshness signal, tune features |
| Slow on specific queries | Expensive wildcards, large aggregations | Profile slow queries, add caching |
| Works locally, breaks in prod | Cold start, cache misses, network | Load testing, warm-up scripts |
| Users always click result #3 | #1 and #2 are ads or irrelevant | Audit top results, check ad ratio |
How to Measure "Good"
Offline Metrics (Before Deployment)
| Precision@K | % of top K results that are relevant |
| Recall@K | % of all relevant docs in top K |
| NDCG | Quality of ranking order (normalized) |
| MRR | How high is the first relevant result? |
Online Metrics (In Production)
| CTR | Are users clicking? (Engagement) |
| Zero Result Rate | % of queries with no results |
| Reformulation | % who search again (frustration) |
| Conversion | Did search lead to purchase/action? |
The Ultimate Test
"If a user searches and leaves without clicking, was your search good or bad?"
Answer: It depends.
✓ Good (no click needed)
User saw the answer in the snippet. "What's the capital of France?" → Paris shown in featured snippet.
✗ Bad (gave up)
User saw irrelevant results and left. Zero engagement → search failed.
This nuance is why search is hard. You can't optimize a single metric.