Implementing Hybrid Semantic + Lexical Search

When Semantic Search Misses the Obvious

Semantic search on kentcdodds.com was an improvement—natural-language questions returned genuinely relevant results. But it had a critical blind spot: searching for "React Testing Library" failed to surface the site's own canonical introduction post for that library. The embedding model weighted the conceptual meaning of the phrase over the literal title match, burying the most obvious result.

That failure triggered three rounds of design, build, ship, and rebuild, each exposing a different class of problem. The final architecture bears little resemblance to the original, and the journey is instructive for anyone running hybrid search on a busy web property.

Closing the Lexical Gap

Vector search excels at intent-based matching. A query like "How do I avoid testing implementation details?" finds relevant posts even without exact keyword overlap. But it routinely fails on exact terms:

• Library names (React Testing Library, Remix, React Router)
• API identifiers (useState, loader, action, useFetcher)
• Error messages, version numbers, names, and product titles

The standard complement is lexical search: build an inverted index and rank with BM25, which rewards documents where query terms appear frequently and prominently.

The obvious answer is to run both retrievers in parallel and merge the candidate lists. The question is where that infrastructure lives.

Round 1: SQLite FTS5 Alongside Vectorize

Cloudflare AI Search was evaluated and rejected: it targets "index my website" scenarios, not custom multi-source pipelines mixing YouTube timestamps, podcast metadata, MDX frontmatter, and per-source ignore lists. The existing Vectorize-based semantic search stayed in place.

The first lexical approach used SQLite FTS5, a built-in extension providing BM25-ranked full-text search with minimal ceremony. The site already had a SQLite cache database replicated across instances via LiteFS—no new infrastructure or billing. The design was threefold: indexers emit lexical artifacts (JSON with chunk text and metadata) to R2; the app syncs those artifacts into a local FTS5 index; at query time, lexical and semantic retrieval run in parallel, merged via Reciprocal Rank Fusion. Development and testing went smoothly. Production revealed the flaw.

The implementation itself was built almost entirely with Cursor and GPT-5.4 over roughly 20 minutes of personal attention, following an agent-generated plan. Before deploy, Cursor's Agent Review caught a subtle regression in YouTube timestamp deep-linking:

// Before the fix
if (!t) return url // t=0 is falsy, so 0-second timestamps were silently dropped

// After the fix
if (t === null) return url // only skip when timestamp is genuinely absent

Transcript chunks at the very start of a video weren't getting the deep-link parameter—a falsy-zero edge case. The agent added a regression test and verified the fix before shipping.

Round 2: Blocking the Request Path

The first production symptom was a novel search query hanging for seconds before returning an HTTP 500. Worse, while that request was in flight, unrelated page loads on the same app instance also hung.

The root cause was architectural: lexical sync logic—fetching JSON artifacts from R2 and importing them into SQLite—was coupled to the foreground search request path. On a cold instance, the first search blocked on that import. Node's DatabaseSync API is synchronous and single-threaded, so any concurrent request touching the same database queued behind it. Because the lexical index lived in the shared cache database, a cold-path sync stalled the entire Node process.

Measurements confirmed the impact:

lexical source replace committed  { sourceKey: "repo-content.json",  durationMs: 71  }
lexical source replace committed  { sourceKey: "podcasts.json",      durationMs: 70  }
lexical source replace committed  { sourceKey: "youtube.json",       durationMs: 75  }
primary lexical sync finished     { durationMs: 273 }

A probing test showed a cold lexical search took ~300ms, while a concurrent "fast" non-search request spent ~160ms blocked behind it. Once warm, that fast request dropped to under 1ms. Derived-index maintenance had no place inside a foreground user request.

Fixing It: A Dedicated Worker

Prewarming on startup, background intervals, and worker threads all kept maintenance on the app server—rejected. The fix moved lexical search off the app server entirely. A dedicated Cloudflare Worker now owns the full lexical stack with its own D1 database for the FTS5 index. The app server queries it over HTTP with a timeout and graceful degradation to semantic-only results on failure. The worker exposes a minimal API: query, sync, and a few admin/delete routes. R2 stays the source of truth; indexing jobs sync artifacts into D1 after they run.

One noteworthy pitfall: the agent's instinct for graceful degradation nearly reintroduced the bug. It left a local SQLite fallback in place, which would have let the blocking problem creep back. The fix removed the local lexical module from the site entirely and switched local development to an MSW-mocked Worker HTTP boundary. Dev and test now exercise the same fetch path as production, intercepted only by the mock.

Round 3: Consolidating into One Search Worker

The split architecture worked but felt wrong. Two separate search stacks meant two deploy workflows, two worker configs, separate env vars, separate monitoring, and separate test surfaces—plus an admin UI for the lexical worker that didn't justify its maintenance cost. The deeper realization: none of the search logic belonged in the app server. The site is a Remix/React Router app whose job is serving pages. Embedding calls, Vectorize queries, and score fusion were there only because that's where the original semantic search happened to live.

Round 3 unified everything behind a single worker with explicit constraints:

• No lexical admin interface
• The site talks to one endpoint, period
• The worker owns embeddings, Vectorize, lexical FTS, and fusion
• Local dev mocks the worker HTTP boundary, running no search logic in-process

Cleaner Inside the Worker

Moving into a Worker simplified infrastructure. The original app-server code called AI Gateway via HTTP with a fetch to gateway.ai.cloudflare.com. Inside a Worker, the Workers AI binding routes through AI Gateway with a single option:

await env.AI.run(
	model,
	{ text: [query] },
	{
		gateway: { id: env.CLOUDFLARE_AI_EMBEDDING_GATEWAY_ID },
	},
)

That eliminates hand-rolled HTTP calls and removes the need for Cloudflare API credentials in the Worker. D1, R2, Vectorize, and AI all become bindings; the only secret is SEARCH_WORKER_TOKEN for callers authenticating to the worker's HTTP boundary.

Shared types and logic (the SearchResult type, error classes, and doc-ID canonicalization) moved into an internal workspace package at services/search-shared (@kcd-internal/search-shared), avoiding long relative imports and making the coupling explicit.

The PR spent several rounds in automated review (CodeRabbit, Cursor BugBot). Actionable feedback focused on hardening: better error handling for malformed JSON in sync requests, memoizing schema initialization per D1 instance, defensive sync body parsing, consistent URL key normalization between worker and shared package, and a test whose assertion was inverted relative to its name. Not architecture changes—but the difference between working code and operationally sound code.

The full diff and review history are in PR #739.

The Production Architecture

The final implementation splits responsibilities across three services: the indexing workers that generate artifacts, a dedicated search worker that handles the full query path, and shared types that keep the contract consistent.

The system produces two distinct kinds of output from its indexing scripts. Vector manifests track what embeddings exist in Vectorize, while separate lexical artifacts—JSON files stored in R2 under keys like lexical-search/repo-content.json—contain the raw searchable text with metadata including title, URL, type, and snippet. For YouTube content, the artifacts also carry startSeconds, endSeconds, and imageUrl.

The Search Worker

services/search-worker is a Cloudflare Worker that owns the runtime query path. It exposes two endpoints:

• POST /search: accepts { query, topK } and returns ranked SearchResult[]
• POST /internal/sync: pulls fresh artifacts from R2 into D1, invoked by GitHub Actions after indexing completes

The worker relies on four Cloudflare bindings: SEARCH_DB (D1, holding the FTS5 lexical index), SEARCH_INDEX (Vectorize, for semantic embeddings), SEARCH_ARTIFACTS_BUCKET (R2, the lexical artifacts), and AI (Workers AI, for generating embeddings).

On a query, the worker fires the FTS lookup and the embedding generation in parallel. Once the embedding resolves, it issues the Vectorize query and fuses the two result sets:

// lexical starts immediately
const lexicalMatchesPromise = dependencies.queryLexicalMatches({
	query,
	topK: rawLexicalTopK,
})

// embedding and lexical run in parallel
const [vector, lexicalMatches] = await Promise.all([
	dependencies.getEmbedding({ text: cleanedQuery, model }),
	lexicalMatchesPromise,
])

// Vectorize query after embedding
const semanticMatches = await dependencies.queryVectorize({
	vector,
	topK: rawSemanticTopK,
})

// fuse both result sets
return fuseRankedResults({ semanticResults, lexicalResults, topK: safeTopK })

Lexical matches receive a 1.15x weight multiplier during fusion versus semantic's 1.0, reflecting that exact-match signals are harder to earn and more trustworthy when present.

The Static Site Boundary

The site itself is now a thin client that delegates entirely to the search worker:

// search.server.ts
const health = await getSearchWorkerHealth().catch(() => null)
const cacheKey = makeSearchCacheKey({
	query: cleanedQuery,
	topK: safeTopK,
	workerUrl,
	searchVersion: health.syncedAt ?? 'never-synced',
})
const baseResults = await cachified({
	key: cacheKey,
	getFreshValue: fetchResults,
})

Result caching keys off syncedAt from the worker's health endpoint. When an indexer run updates the lexical index, the cache automatically invalidates without any explicit purge mechanism. The site performs no embedding, vector querying, or score fusion on its own—it proxies to the worker and caches the result.

The Shared Contract

services/search-shared (published as @kcd-internal/search-shared) exports the SearchResult type, SearchQueryTooLongError, the normalizeSearchQuery helper, and the canonical doc-ID logic. Both the worker and the site import from this package, which prevents contract drift without resorting to fragile relative imports.

Confidence Filtering

After RRF fusion, the worker runs results through filterFusedResultsByConfidence in services/search-worker/src/search-results.ts. Two thresholds govern this step: SEARCH_CONFIDENCE_MIN_BEST_SCORE (default 0.013) rejects queries whose top fused score falls below the floor, and SEARCH_CONFIDENCE_RELATIVE_RATIO (default 0.5) drops any hit scoring below half of the top score, provided the top hit cleared the floor. The /search response includes noCloseMatches: true when candidates existed but all were filtered out—a distinct state from an empty index. Tuning guidance lives in docs/agents/search-relevance.md.

Operational Lessons

Search quality was not the hard part. Exact terms like "React Testing Library" and API identifiers reliably surfaced the right content from the first implementation onward.

The real friction was operational. Foremost, derived-index maintenance must never run inside a user-facing request. Keeping a search index fresh is background work, regardless of whether the backing store is SQLite or a managed service. Splitting the system across two worker contexts—each with its own deploy pipeline, environment variables, and test surface—works, but compounds operational overhead. When a feature crosses two execution boundaries, it's worth asking whether that split is genuinely required or merely accidental.

The most significant shift came from AI-assisted development. Three full redesign cycles happened in days because an agent handled the mechanical implementation, leaving the human to focus on design judgment. When the cost of restructuring drops that low, acting on second thoughts about an architecture becomes routine rather than something to live with out of sunk-cost inertia.